samples: net: Unify certificate handling

Update NET samples to be able to import and use certificates
in valid PEM format.

Signed-off-by: Simen S. Røstad <simen.rostad@nordicsemi.no>

applications/asset_tracker_v2/CMakeLists.txt

@@ -30,16 +30,19 @@ add_subdirectory_ifdef(CONFIG_WATCHDOG_APPLICATION src/watchdog)
 # Include nRF modem library header file for PC builds.
 # These are used throughout the application in type definitions.
 if (CONFIG_BOARD_QEMU_X86 OR CONFIG_BOARD_NATIVE_POSIX)
-        target_include_directories(app PRIVATE ${NRFXLIB_DIR}/nrf_modem/include/)
+	target_include_directories(app PRIVATE ${NRFXLIB_DIR}/nrf_modem/include/)
 
-        # Make folder containing certificates global so that it can be located by the configured
-        # cloud library.
-        zephyr_include_directories(src/cloud-certs)
+	# Make the folder that contains the certificates global so that it can be located by the
+	# nRF Cloud library. For the other clouds integrations, the certificates are handled by the
+	# MQTT helper library, see CONFIG_MQTT_HELPER_PROVISION_CERTIFICATES.
+	if (CONFIG_NRF_CLOUD_PROVISION_CERTIFICATES)
+		zephyr_include_directories(src/cloud-certs)
+	endif()
 
-        target_compile_options(app PRIVATE
-	        -DCONFIG_LTE_NEIGHBOR_CELLS_MAX=10)
+	target_compile_options(app PRIVATE
+		-DCONFIG_LTE_NEIGHBOR_CELLS_MAX=10)
 endif()
 
 if(CONFIG_ASSET_TRACKER_V2_LTO)
-        target_compile_options(app PRIVATE "-flto")
+	target_compile_options(app PRIVATE "-flto")
 endif()

applications/asset_tracker_v2/boards/native_sim.conf

@@ -116,3 +116,6 @@ CONFIG_BUILD_S1_VARIANT=n
 
 # Watchdog
 CONFIG_WATCHDOG_APPLICATION=n
+
+# Cloud module certificates
+CONFIG_MQTT_HELPER_CERTIFICATES_FOLDER="src/cloud-certs"

doc/nrf/libraries/networking/aws_iot.rst

@@ -188,9 +188,9 @@ There are multiple ways to generate and register these certificates:
 
       #. Take note of the certificate ARN, as it will be required later.
       #. Download the `Amazon Root CA 1`_ PEM file as :file:`ca-cert.pem`.
-      #. Provision the certificates and private key at runtime to the Mbed TLS stack.
-         This is achieved by placing the PEM files into a :file:`certs/` subdirectory and ensuring the :kconfig:option:`CONFIG_MQTT_HELPER_PROVISION_CERTIFICATES` Kconfig option is enabled.
-         For more information, refer to the :ref:`aws_iot` sample as well as the :kconfig:option:`CONFIG_MQTT_HELPER_CERTIFICATES_FILE` Kconfig option.
+      #. Place the PEM files into the folder path specified by the :kconfig:option:`CONFIG_MQTT_HELPER_CERTIFICATES_FOLDER` option, default is :file:`<app_src_dir>/certs/`.
+         Ensure that the :kconfig:option:`CONFIG_MQTT_HELPER_PROVISION_CERTIFICATES` option is set.
+      #. If the files are placed correctly, the :ref:`lib_mqtt_helper` library finds the certificates and provisions them to the Mbed TLS stack when connecting to AWS IoT.
 
 .. rst-class:: numbered-step
 

doc/nrf/libraries/networking/azure_iot_hub.rst

@@ -342,14 +342,6 @@ The following are the ways to generate and register device certificates:
 
       #. Provision the certificates and private key at runtime to the Mbed TLS stack.
          This is achieved by placing the PEM files into a :file:`certs/` subdirectory and ensuring the :kconfig:option:`CONFIG_MQTT_HELPER_PROVISION_CERTIFICATES` Kconfig option is enabled.
-         The PEM files need to be converted to string format to be compiled into the firmware.
-
-         To do this, enclose each line in the PEM file with double quotes and finish each line with a newline character, using the following command:
-
-         .. code-block:: console
-
-            sed -i'.org' 's/.*/"&\\n"/' <pem_file>
-
          For more information, refer to the :ref:`azure_iot_hub` sample as well as the :kconfig:option:`CONFIG_MQTT_HELPER_CERTIFICATES_FILE` Kconfig option.
 
 .. rst-class:: numbered-step

doc/nrf/libraries/networking/mqtt_helper.rst

@@ -27,7 +27,7 @@ Additionally, configure the following options as per the needs of your applicati
 * :kconfig:option:`CONFIG_MQTT_HELPER_RX_TX_BUFFER_SIZE`
 * :kconfig:option:`CONFIG_MQTT_HELPER_PAYLOAD_BUFFER_LEN`
 * :kconfig:option:`CONFIG_MQTT_HELPER_PROVISION_CERTIFICATES`
-* :kconfig:option:`CONFIG_MQTT_HELPER_CERTIFICATES_FILE`
+* :kconfig:option:`CONFIG_MQTT_HELPER_CERTIFICATES_FOLDER`
 
 API documentation
 *****************

doc/nrf/releases_and_maturity/migration/migration_guide_2.7.rst

@@ -31,7 +31,16 @@ Samples and applications
 
 This section describes the changes related to samples and applications.
 
-|no_changes_yet_note|
+* For applications using the :ref:`lib_mqtt_helper` library:
+
+  * The ``CONFIG_MQTT_HELPER_CERTIFICATES_FILE`` is now replaced by :kconfig:option:`CONFIG_MQTT_HELPER_CERTIFICATES_FOLDER`.
+    The new option is a folder path where the certificates are stored.
+    The folder path must be relative to the root of the project.
+
+    If you are using the :ref:`lib_mqtt_helper` library, you must update the Kconfig option to use the new option.
+
+  * When using the :kconfig:option:`CONFIG_MQTT_HELPER_PROVISION_CERTIFICATES` Kconfig option, the certificate files must be in standard PEM format.
+    This means that the PEM files must not be converted to string format anymore.
 
 Libraries
 =========

doc/nrf/releases_and_maturity/releases/release-notes-changelog.rst

@@ -243,7 +243,9 @@ Multicore samples
 Networking samples
 ------------------
 
-|no_changes_yet_note|
+* Updated:
+
+  *  The networking samples to support import of certificates in valid PEM formats.
 
 NFC samples
 -----------
@@ -356,6 +358,11 @@ Libraries for networking
     * The :kconfig:option:`CONFIG_NRF_CLOUD_LOCATION_ANCHOR_LIST` Kconfig option to enable including Wi-Fi anchor names in the location callback.
     * The :kconfig:option:`CONFIG_NRF_CLOUD_LOCATION_ANCHOR_LIST_BUFFER_SIZE` Kconfig option to control the buffer size used for the anchor names.
 
+* :ref:`lib_mqtt_helper` library:
+
+  * Changed the library to read certificates as standard PEM format. Previously the certificates had to be manually converted to string format before compiling the application.
+  * Replaced the ``CONFIG_MQTT_HELPER_CERTIFICATES_FILE`` Kconfig option with :kconfig:option:`CONFIG_MQTT_HELPER_CERTIFICATES_FOLDER`. The new option specifies the folder where the certificates are stored.
+
 Libraries for NFC
 -----------------
 

samples/matter/light_bulb/overlay-aws-iot-integration.conf

@@ -20,6 +20,7 @@ CONFIG_JSON_LIBRARY=y
 
 # MQTT helper library
 CONFIG_MQTT_HELPER_PROVISION_CERTIFICATES=y
+CONFIG_MQTT_HELPER_CERTIFICATES_FOLDER="src/aws_iot_integration/certs"
 CONFIG_MQTT_HELPER_SEC_TAG=201
 
 # MQTT - Maximum MQTT keepalive timeout specified by AWS IoT Core

samples/matter/light_bulb/src/aws_iot_integration/CMakeLists.txt

@@ -7,10 +7,6 @@
 target_include_directories(app PRIVATE ${CMAKE_CURRENT_SOURCE_DIR})
 
 if (CONFIG_AWS_IOT_INTEGRATION)
-	zephyr_include_directories(
-		CONFIG_MQTT_HELPER_PROVISION_CERTIFICATES certs
-	)
-
 	target_include_directories(app PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/codec)
 
 	target_sources(app PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/aws_iot_integration.c)

samples/net/aws_iot/CMakeLists.txt

@@ -16,6 +16,3 @@ target_sources(app PRIVATE src/json_payload/json_payload.c)
 
 zephyr_include_directories(src)
 zephyr_include_directories(src/json_payload)
-
-# Make folder containing certificates global so that it can be located by the MQTT helper library.
-zephyr_include_directories_ifdef(CONFIG_MQTT_HELPER_PROVISION_CERTIFICATES certs)

samples/net/aws_iot/README.rst

@@ -151,12 +151,12 @@ The sample includes pre-configured configuration files for the development kits
 * :file:`boards/nrf9161dk_nrf9161_ns.conf` - Configuration file for the nRF9161 DK.
 * :file:`boards/nrf9160dk_nrf9160_ns.conf` - Configuration file for the nRF9160 DK.
 * :file:`boards/thingy91_nrf9160_ns.conf` - Configuration file for the Thingy:91.
-* :file:`boards/nrf7002dk_nrf5340_cpuapp.conf` - Configuration file for the nRF7002 DK.
+* :file:`boards/nrf7002dk_nrf5340_cpuapp_ns.conf` - Configuration file for the nRF7002 DK.
 * :file:`boards/qemu_x86.conf` - Configuration file for QEMU x86.
 
 The following configuration and DTS overlay files are included to host the MCUboot secondary image slot on external flash for the nRF7002 DK:
 
-* :file:`boards/nrf7002dk_nrf5340_cpuapp.overlay` - DTS overlay file for the application image.
+* :file:`boards/nrf7002dk_nrf5340_cpuapp_ns.overlay` - DTS overlay file for the application image.
 * :file:`child_image/mcuboot/nrf7002dk_nrf5340_cpuapp.overlay` - DTS overlay file for the MCUboot child image.
 * :file:`child_image/mcuboot/nrf7002dk_nrf5340_cpuapp.conf` - Configuration file for the MCUboot child image.
 

samples/net/azure_iot_hub/CMakeLists.txt

@@ -15,7 +15,3 @@ target_sources(app PRIVATE src/main.c)
 # NORDIC SDK APP END
 
 zephyr_include_directories(src)
-
-# Make folder containing certificates global so that it can be located by
-# the MQTT helper library.
-zephyr_include_directories_ifdef(CONFIG_MQTT_HELPER_PROVISION_CERTIFICATES certs)

samples/net/azure_iot_hub/boards/native_sim.conf

@@ -53,4 +53,3 @@ CONFIG_NET_CONFIG_MY_IPV4_GW="192.0.2.2"
 
 # Azure IoT Hub library
 CONFIG_MQTT_HELPER_SEC_TAG=201
-CONFIG_MQTT_HELPER_SECONDARY_SEC_TAG=-1

samples/net/coap_client/boards/nrf7002dk_nrf5340_cpuapp_ns.conf

@@ -41,6 +41,7 @@ CONFIG_SHELL_STACK_SIZE=6144
 
 # WPA
 CONFIG_WPA_SUPP=y
+CONFIG_MBEDTLS_HEAP_SIZE=16384
 
 # NET sockets
 CONFIG_NET_L2_ETHERNET=y
@@ -56,12 +57,3 @@ CONFIG_NET_SOCKETS_DNS_TIMEOUT=30000
 CONFIG_L2_WIFI_CONNECTIVITY=y
 CONFIG_L2_WIFI_CONNECTIVITY_AUTO_DOWN=n
 CONFIG_L2_WIFI_CONNECTIVITY_AUTO_CONNECT=n
-
-# TLS
-CONFIG_NRF_SECURITY=y
-CONFIG_MBEDTLS=y
-CONFIG_MBEDTLS_ENABLE_HEAP=y
-CONFIG_MBEDTLS_HEAP_SIZE=16384
-CONFIG_MBEDTLS_RSA_C=y
-CONFIG_MBEDTLS_SSL_SERVER_NAME_INDICATION=y
-CONFIG_NET_SOCKETS_SOCKOPT_TLS=y

samples/net/http_server/CMakeLists.txt

@@ -12,4 +12,33 @@ project(http_server)
 target_sources(app PRIVATE src/main.c)
 target_sources(app PRIVATE src/credentials_provision.c)
 target_include_directories(app PRIVATE src)
-target_include_directories(app PRIVATE credentials)
+
+if (CONFIG_NET_SOCKETS_SOCKOPT_TLS)
+	message(WARNING "Credentials are exposed in non-secure memory. This should be avoided in production.")
+
+	set(app_certs_binary_dir ${APPLICATION_BINARY_DIR}/credentials)
+
+	# Function to process a certificate file and create a corresponding .inc file and compiler definition
+	# used to assign C variables that are used in main.c.
+	function(process_certificate definition_name file_name)
+		set(cert_file ${APPLICATION_SOURCE_DIR}/credentials/${file_name})
+		if(EXISTS ${cert_file})
+			message(STATUS "${file_name} found")
+
+			get_filename_component(file_base_name ${file_name} NAME_WE)
+			set(inc_file_name ${file_base_name}.inc)
+
+			set(inc_file_path ${app_certs_binary_dir}/${inc_file_name})
+			generate_inc_file_for_target(app ${cert_file} ${inc_file_path})
+
+			# Define a compiler macro with the path to the generated .inc file,
+			# allowing it to be included in the source code.
+			add_definitions(-D${definition_name}="${inc_file_path}")
+		endif()
+	endfunction()
+
+	# Process each certificate file by generating a .inc file and defining a corresponding macro.
+	process_certificate("HTTP_SERVER_CA_CERT" "server_certificate.pem")
+	process_certificate("HTTP_SERVER_PRIVATE_KEY" "server_private_key.pem")
+
+endif()

samples/net/http_server/README.rst

@@ -137,14 +137,8 @@ To generate new credentials, run the following commands:
          # Sign the client's CSR with the server's private key and certificate, creating a client certificate.
          openssl x509 -req -in client.csr -CA server_certificate.pem -CAkey server_private_key.pem -CAcreateserial -out client.crt -days 365
 
-To provision the generated credentials to the server's TLS stack, the credentials need to be converted into C header files that are readable by the server code.
-To do this, you can use the following script :file:`sdk-nrf/scripts/cert_to_header.py`.
-Replace the generated header files with the pregenerated header files in the :file:`http_server/credentials` folder.
-
-.. code-block:: console
-
-   python3 cert_to_header.py server_certificate.pem
-   python3 cert_to_header.py server_private_key.pem
+To provision the generated credentials to the server's TLS stack, replace the pregenerated certificates with the newly generated one in the :file:`http_server/credentials` folder in PEM format.
+Provisioning happens automatically after the firmware boots by the sample.
 
 Configuration
 *************

samples/net/http_server/boards/nrf7002dk_nrf5340_cpuapp_ns.conf

@@ -46,9 +46,6 @@ CONFIG_MBEDTLS_HEAP_SIZE=16384
 # DHCPv4
 CONFIG_NET_DHCPV4=y
 
-# TLS credentials
-CONFIG_TLS_CREDENTIALS_BACKEND_PROTECTED_STORAGE=y
-
 # Shell and Wi-fi Credentials
 CONFIG_SHELL=y
 CONFIG_NET_SHELL=y

samples/net/http_server/overlay-tls-nrf7002dk.conf

@@ -12,3 +12,6 @@ CONFIG_NET_SOCKETS_TLS_MAX_CONTEXTS=3
 CONFIG_NORDIC_SECURITY_BACKEND=y
 CONFIG_MBEDTLS_HEAP_SIZE=81920
 CONFIG_MBEDTLS_RSA_C=y
+
+# TLS credentials
+CONFIG_TLS_CREDENTIALS_BACKEND_PROTECTED_STORAGE=y

samples/net/http_server/overlay-tls-nrf91.conf

@@ -57,6 +57,8 @@ CONFIG_MBEDTLS_KEY_EXCHANGE_ECDHE_RSA_ENABLED=y
 CONFIG_MBEDTLS_KEY_EXCHANGE_RSA_ENABLED=y
 CONFIG_MBEDTLS_PKCS1_V15=y
 CONFIG_MBEDTLS_MPI_MAX_SIZE=512
+CONFIG_PSA_WANT_KEY_TYPE_RSA_PUBLIC_KEY=y
+CONFIG_PSA_WANT_RSA_KEY_SIZE_2048=y
 
 # Enable SHA
 CONFIG_MBEDTLS_SHA256_C=y