Newer GNU bin-utils version 2.39 has started emitting a warning
about an executable stack region for Linux target build:
/usr/bin/ld: warning: x509_crt_bundle.S.o: missing .note.GNU-stack section implies executable stack
Fix the issue by adding appropriate "noexecute" attribute for stack
section.
- Example CA certificates that are used for self-signed client certificates
need to include the Key Usage parameter.
- Python3.13 changed the default context of the SSL context that is
generated using ssl.create_default_context() by enabling the VERIFY_X509_STRICT
flag by default
For the Linux target, we currently attempt to fallback to older C/CXX
lagnuage standards in the __build_set_lang_version() function. The
language standard support is checked using CMake's language-specific
functions, such as check_c_compiler_flag(). These functions require the
language to be enabled[1] in CMake beforehand, which is done by calling
project() or by enabling the languages later with enable_language(). At
present, we use enable_language() to enable C and CXX languages in
CMake, allowing us to set the standard early, before invoking project().
However, newer CMake versions (>3.29) issue a warning[2] if
enable_language() is called before project(), as noted in CMP0165[3].
It should generally be acceptable to call __build_set_lang_version()
after __project(), but doing so would alter the behavior of the
COMPILE_OPTIONS also for non-Linux targets. Currently, users can
add to COMPILE_OPTIONS even before calling project() in the project's
CMakeLists.txt and the options will be in the desired order. In other
words, appending to COMPILE_OPTIONS can occur either before or after
calling project() in the project's CMakeLists.txt, with the outcome
remaining consistent. This means the user's settings will appear later
and take priority. However, if __build_set_lang_version() is called
after __project(), the user's COMPILE_OPTIONS settings would be
overridden if set before calling project(). Our documentation[4] explicitly
states that COMPILE_OPTIONS and similar properties should be modified
using idf_build_set_property() after calling project() to prevent
default values from overwriting them.
Even with this guidance, some existing components that modify
COMPILE_OPTIONS before invoking project() might be impacted by this
change. Therefore, separate the language standard settings for non-Linux
and Linux targets. For non-Linux targets, these settings are applied in
__build_set_default_build_specifications(), maintaining the current
behavior. For the Linux target, the language standard is set with
__linux_build_set_lang_version() after calling __project(), ensuring the
languages are already enabled in CMake and no warning is issued. Since the Linux
target is still in preview, this approach should be acceptable,
especially with the existing documentation[4].
Closes https://github.com/espressif/esp-idf/issues/15488
[1] https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#languages
[2] https://gitlab.kitware.com/cmake/cmake/-/merge_requests/9396
[3] https://cmake.org/cmake/help/latest/policy/CMP0165.html#policy:CMP0165
[4] https://docs.espressif.com/projects/esp-idf/en/v5.4/esp32/api-guides/
build-system.html#overriding-default-build-specifications
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `setuptools` package starting with `v70.1.0`[1] contains built-in
`bdist_wheel` command. Before this version `setuptools` relied on the
`bdist_wheel` command implementation from the `wheel` package. Starting with
`setuptools` `v75.8.1` the `PEP 491`[3] restrictions on the distribution name
of a wheel package are enforced[4], replacting also `.` with `_`. Note that
`PEP 491` actually allows `.` in the distribution name, but for some reason the
latest packaging docs[10][11] does not, stating that `.` should be replaced
with `_`. This was discussion here[12].
Also the `wheel` package starting with `v0.45.0`[5] is using the `bdist_wheel`
command from `setuptools`. This means that any package which has `.` in its
distribution name, like `ruamel.yaml.clib`, can have different wheel name,
depending on which version of the `bdist_wheel` command was used.
The `bdist_wheel` command from setuptools prior `v75.8.1` or `wheel` prior
`v0.45.0` will keep the dots in distribution name preserved. For exaple the
`ruamel.yaml.clib` package will have distribution name
`ruamel.yaml.clib-0.2.12.dist-info. Newer versions will replace the dots with
`_` according to [10][11], creating distribution like
`ruamel_yaml_clib-0.2.12.dist-info`.
From packaging point of view `ruamel.yaml.clib-0.2.12.dist-info` and
`ruamel_yaml_clib-0.2.12.dist-info` are the same packages, but this is not
reflected in `importlib.metadata` prior python 3.10[9], which does not perform
name normalization prior the distribution search. This causes the `version`
from `importlib.metadata` to fail on python prior the 3.10 version if the
package with dots in distribution name was generated with normalized paths with
newer `setuptools`. Note that the distribution name normalization was
backported to some later 3.9 python version.
Let's demonstrate this behavior on a simple package with the
`my.minimal.package` name.
```
my_minimal_package/
├── pkg
│ └── __init__.py
└── setup.py
from setuptools import setup, find_packages
setup(
name='my.minimal.package',
version='0.1.0',
packages=find_packages(),
install_requires=[],
entry_points={},
)
```
With python 3.9.0 search for `my.minimal.package` fails because
of the missing name normalization.
```
docker run --rm -it --platform linux/x86_64 python:3.9.0 bash
python -m venv venv
. venv/bin/activate
pip install setuptools==v75.8.1
python setup.py bdist_wheel
pip install dist/my_minimal_package-0.1.0-py3-none-any.whl
python
Python 3.9.0 (default, Nov 18 2020, 13:28:38)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib.metadata import version as get_version
>>> get_version('my.minimal.package')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.9/importlib/metadata.py", line 551, in version
return distribution(distribution_name).version
File "/usr/local/lib/python3.9/importlib/metadata.py", line 524, in distribution
return Distribution.from_name(distribution_name)
File "/usr/local/lib/python3.9/importlib/metadata.py", line 187, in from_name
raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: my.minimal.package
>>> get_version('my_minimal_package')
'0.1.0'
```
With python 3.10.0 search for both `my.minimal.package` and
`my_minimal_package` succeeds.
```
docker run --rm -it --platform linux/x86_64 python:3.10.0 bash
python
Python 3.10.0 (default, Dec 3 2021, 00:21:30) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib.metadata import version as get_version
>>> get_version('my.minimal.package')
'0.1.0'
>>> get_version('my_minimal_package')
'0.1.0'
```
In our `tools/check_python_dependencies.py` we cannot relay on the default
distribution finder, used in the `version` function from `importlib.metadata`,
to do name normalization on older python versions. To cope with this,
implement a fallback version search. If `version` fails with
`PackageNotFoundError`, do the name normalization according to [10][11] and try
again.
Note: There is also a `wheel`[6][7] `v0.43.0` package embeded in `setuptools`
along with the new implementation[8]. This one seems to be used if the
external `wheel` package is not available but imported. TBH this is all kinda
messy and I may have overlooked something.
* [1] https://setuptools.pypa.io/en/stable/history.html#v70-1-0
* [2] https://setuptools.pypa.io/en/stable/history.html#v75-8-1
* [3] https://peps.python.org/pep-0491/#escaping-and-unicode
* [4] https://github.com/pypa/setuptools/pull/4766/files
* [5] https://wheel.readthedocs.io/en/stable/news.html
* [6] https://github.com/pypa/setuptools/blob/main/setuptools/_vendor/wheel/__init__.py
* [7] https://github.com/pypa/setuptools/issues/1386
* [8] https://github.com/pypa/setuptools/blob/main/setuptools/command/bdist_wheel.py
* [9] c6ca368867
* [10] https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization
* [11] https://packaging.python.org/en/latest/specifications/binary-distribution-format/
#escaping-and-unicode
* [12] https://github.com/pypa/setuptools/issues/3777
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Using same IV in AES-GCM across multiple invocation of
encryption/decryption operations can pose a security risk. It can help
to reveal co-relation between different plaintexts.
This commit introduces a change to use part of IV as a monotonic
counter, which must be incremented after every AES-GCM invocation
on both the client and the device side.
Concept of patch version for a security scheme has been introduced here
which can help to differentiate a protocol behavior for the provisioning
entity. The security patch version will be available in the JSON
response for `proto-ver` endpoint request with the field
`sec_patch_ver`.
Please refer to documentation for more details on the changes required
on the provisioning entity side (e.g., PhoneApps).
This commit updates the following:
- Updates the panic handler to use only the RTC WDT to reset the system.
- Refactors some of the panic handler code.
- Updates Bluetooth files where in they now feed the WDTs instead of
reconfiguring them.
- Removes some unnecessary configuration of WDTs from various files.
- Added a unit test to verify that the system does not lock up when the
panic handler is stuck.
- Updates the memprot unit tests to work with the refactored panic
handler.
Closes https://github.com/espressif/esp-idf/issues/15166
Closes https://github.com/espressif/esp-idf/issues/15018
Closes https://github.com/espressif/esp-idf/issues/10110
The active.py script is currently clearing the IDF_PYTHON_ENV_PATH,
preventing it from being set to a custom location for the python virtual
environment directory. Although the install script checks to ensure that
an existing python virtual environment is not overwritten with one for a
different ESP-IDF version than it was originally created for, we should
still permit setting a custom path for the python virtual environment.
Closes https://github.com/espressif/esp-idf/issues/15006
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When a symbol needs to be placed to a different target than the one
designated for the object file, the object file is expanded, which
includes the following steps:
1. Creating a new placement for the symbol's input section with the
specified target.
2. Excluding the object placement from the orignal target.
3. Creating a new intermediate placement for the object for the original
target, where its input sections are expanded, excluding the input
section for the symbol.
Let's illustrate the object expansion process with the following example:
[sections:rodata]
entries:
.rodata+
.sdata2+
.srodata+
[scheme:default]
entries:
text -> flash_text
rodata -> flash_rodata
[scheme:noflash]
entries:
text -> iram0_text
rodata -> dram0_data
[mapping:soc_pm]
archive: libsoc.a
entries:
gpio_periph: GPIO_HOLD_MASK (noflash)
gpio_periph section headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .text PROGBITS 00000000 000034 000000 00 AX 0 0 2
[ 2] .data PROGBITS 00000000 000034 000000 00 WA 0 0 1
[ 3] .bss NOBITS 00000000 000034 000000 00 WA 0 0 1
[ 4] .rodata.GPIO_HOLD_MASK PROGBITS 00000000 000034 000058 00 A 0 0 4
[ 5] .rodata.GPIO_PIN_MUX_REG PROGBITS 00000000 00008c 000058 00 A 0 0 4
[ 6] .debug_info PROGBITS 00000000 0000e4 0000d8 00 0 0 1
[ 7] .rela.debug_info RELA 00000000 0009d4 000108 0c I 16 6 4
[ 8] .debug_abbrev PROGBITS 00000000 0001bc 000070 00 0 0 1
[ 9] .debug_aranges PROGBITS 00000000 00022c 000018 00 0 0 1
[10] .rela.debug_aranges RELA 00000000 000adc 00000c 0c I 16 9 4
[11] .debug_line PROGBITS 00000000 000244 0001ab 00 0 0 1
[12] .debug_str PROGBITS 00000000 0003ef 00022d 01 MS 0 0 1
[13] .comment PROGBITS 00000000 00061c 000030 01 MS 0 0 1
[14] .note.GNU-stack PROGBITS 00000000 00064c 000000 00 0 0 1
[15] .riscv.attributes RISCV_ATTRIBUTES 00000000 00064c 000044 00 0 0 1
[16] .symtab SYMTAB 00000000 000690 000260 10 17 36 4
[17] .strtab STRTAB 00000000 0008f0 0000e1 00 0 0 1
[18] .shstrtab STRTAB 00000000 000ae8 0000d1 00 0 0 1
1. Creating a new placement
.dram0.data :
{
*libsoc.a:gpio_periph.*(.rodata.GPIO_HOLD_MASK .sdata2.GPIO_HOLD_MASK .srodata.GPIO_HOLD_MASK)
}
2. Excluding the object placement
.flash.rodata :
{
*(EXCLUDE_FILE(*libsoc.a:gpio_periph.*) .rodata.* ...)
}
3. Creating a new intermediate placement
.flash.rodata :
{
*libsoc.a:gpio_periph.*(.rodata.GPIO_PIN_MUX_REG)
}
Now, let's do the same, but also move GPIO_PIN_MUX_REG to noflash with an updated mapping.
[mapping:soc_pm]
archive: libsoc.a
entries:
gpio_periph: GPIO_HOLD_MASK (noflash)
gpio_periph: GPIO_PIN_MUX_REG (noflash)
1. Creating a new placement
.dram0.data :
{
*libsoc.a:gpio_periph.*(.rodata.GPIO_HOLD_MASK .sdata2.GPIO_HOLD_MASK .srodata.GPIO_HOLD_MASK)
*libsoc.a:gpio_periph.*(.rodata.GPIO_PIN_MUX_REG .sdata2.GPIO_PIN_MUX_REG
.srodata.GPIO_PIN_MUX_REG)
}
2. Excluding the object placement
.flash.rodata :
{
*(EXCLUDE_FILE(*libsoc.a:gpio_periph.*) .rodata.* ...)
}
3. Creating a new intermediate placement
.flash.rodata :
{
*libsoc.a:gpio_periph.*
}
The *libsoc.a:gpio_periph.* entity in step 3 no longer has input
sections, as there are no remaining .rodata input sections in the object
file. The linker behavior for this mapping is to include all object
input sections that have not yet been placed as described in
https://sourceware.org/binutils/docs/ld.html#Input-Section-Basics
"If you use a file name without a list of sections, then all sections in
the input file will be included in the output section. This is not
commonly done, but it may by useful on occasion."
The map file for such mapping now contains following input sections
.flash.rodata 0x3c0a0120 0x19b34
*libsoc.a:gpio_periph.*()
.debug_info 0x3c0b95bf 0xd8 esp-idf/soc/libsoc.a(gpio_periph.c.obj)
.debug_abbrev 0x3c0b9697 0x70 esp-idf/soc/libsoc.a(gpio_periph.c.obj)
.debug_aranges
0x3c0b9707 0x18 esp-idf/soc/libsoc.a(gpio_periph.c.obj)
.debug_line 0x3c0b971f 0x1ab esp-idf/soc/libsoc.a(gpio_periph.c.obj)
.debug_str 0x3c0b98ca 0x21a esp-idf/soc/libsoc.a(gpio_periph.c.obj)
0x22d (size before relaxing)
.comment 0x3c0b9ae4 0x30 esp-idf/soc/libsoc.a(gpio_periph.c.obj)
.note.GNU-stack
0x3c0b9ae4 0x0 esp-idf/soc/libsoc.a(gpio_periph.c.obj)
.riscv.attributes
0x3c0b9ae4 0x44 esp-idf/soc/libsoc.a(gpio_periph.c.obj)
This is incorrect, and such intermediate placement should not be
generated. This type of placement can be recognized because it is not
explicitly defined in the mapping and lacks input sections. We can
identify this in the significant function and prevent issuing commands
for such placement.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
