From 1a827a342219f84e55adb24ca5facfef517952f3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Mon, 13 Nov 2023 10:01:21 +0100 Subject: [PATCH 01/20] Start documenting test-driver framework. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- .../testing/driver-interface-test-strategy.md | 105 +++++++++++++++++- 1 file changed, 103 insertions(+), 2 deletions(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index 380fd39c43..707b6a64a3 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -114,7 +114,7 @@ We should have at least one driver that covers the whole interface: A PKCS#11 driver would be a good candidate. It would be useful as part of our product offering. -## Transparent driver interface testing +## Unified driver interface testing The [unified driver interface](../../proposed/psa-driver-interface.md) defines interfaces for accelerators. @@ -128,6 +128,107 @@ Every cryptographic mechanism for which a transparent driver interface exists (k The driver interface includes a fallback mechanism so that a driver can reject a request at runtime and let another driver handle the request. For each entry point, there must be at least three test runs with two or more drivers available with driver A configured to fall back to driver B, with one run where A returns `PSA_SUCCESS`, one where A returns `PSA_ERROR_NOT_SUPPORTED` and B is invoked, and one where A returns a different error and B is not invoked. -## Entropy and randomness interface testing +### Test framework + +We have test drivers that are enabled by `PSA_CRYPTO_DRIVER_TEST` (not present +in the usual config files, must be defined on the command line or in a custom +config file). Those test drivers are implemented in `tests/src/drivers/*.c` +and their API is declared in `tests/include/test/drivers/*.h`. + +We have two test driver registered: `mbedtls_test_opaque_driver` and +`mbedtls_test_transparent_driver`. These are described in +`scripts/data_files/driver_jsons/mbedtls_test_xxx_driver.json` (as much as our +JSON support currently allows). Each of the drivers can potentially implement +support for several mechanism; conversely, each of the file mentioned in the +previous paragraph can potentially contribute to both the opaque and the +transparent test driver. + +Each entry point is instrumented to record the number of hits for each part of +the driver (same division as the files) and the status of the last call. It is +also possible to force the next call to return a specified status. See the +various `mbedtls_test_driver_XXX_hooks_t` structures declared by each driver. + +The drivers can use one of two back-ends: +- internal: this requires the built-in implementation to be present. +- libtestdriver1: this allows the built-in implementation to be omitted from + the build. + +Historical note: internal was initially the only back-end; then support for +libtestdriver1 was added gradually. + +Question: if/when we have complete libtestdriver1 support, do we still need +internal? Thoughts: +- It's useful to have builds with both a driver and the built-in, in +order to test fallback to built-in, but this could be achieved with +libtestdriver1 too. + - Performance might be better with internal though? +- The instrumentation works the same with both back-ends. + +Our implementation of PSA Crypto is structured in a way that the built-in +implementation of each operation follows the driver API, see +[`../architecture/psa-crypto-implementation-structure.md`](../architecture/psa-crypto-implementation-structure.html). +This makes implementing the test drivers very easy: each entry point has a +corresponding `mbedtls_psa_xxx()` function that it can call as its +implementation - with the `libtestdriver1` back-end the function is called +`libtestdriver1_mbedtls_psa_xxx()` instead. + +The renaming process for `libtestdriver1` is implemented as a few Perl regexes +applied to a copy of the library code, see the `libtestdriver1.a` target in +`tests/Makefile`. Another modification that's done to this copy is appending +`tests/include/test/drivers/crypto_config_test_driver_extension.h` to +`psa/crypto_config.h`. This file reverses the `ACCEL`/`BUILTIN` macros so that +`libtestdriver1` includes as built-in what the main `libmbedcrypto.a` will +have accelerated; see that file's initial comment for details. See also +`helper_libtestdriver1_` functions and the preceding comment in `all.sh` for +how libtestdriver is used in practice. + +This general framework needs specific code for each family of operations. At a +given point in time, not all operations have the same level of support. The +following sub-sections describe the status of the test driver support, mostly +following the structure and order of sections 9.6 and 10.2 to 10.10 of the +[PSA Crypto standard](https://arm-software.github.io/psa-api/crypto/1.1/) as +that is also a natural division for implementing test drivers (that's how the +code is divided into files). It should be noted that the implementation +strategy ensures that when and entry point has test-driver support, it +automatically works for all algorithms and key types supported by the library, +thanks to the implementation strategy mentioned above. + +#### Key management + +TODO + +#### Message digests (Hashes) + +TODO + +#### Message authentication codes (MAC) + +TODO + +#### Unauthenticated ciphers + +TODO + +#### Authenticated encryption with associated data (AEAD) + +TODO + +#### Key derivation + +TODO + +#### Asymmetric signature + +TODO + +#### Asymmetric encryption + +TODO + +#### Key agreement + +TODO + +#### Other cryptographic services (Random number generation) TODO From b66f9dba119e6860e95020e3fd8ada4bd496286d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Mon, 13 Nov 2023 11:32:37 +0100 Subject: [PATCH 02/20] Document test-driver status per family MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- .../testing/driver-interface-test-strategy.md | 214 ++++++++++++++++-- 1 file changed, 197 insertions(+), 17 deletions(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index 707b6a64a3..7c43a4feba 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -128,7 +128,7 @@ Every cryptographic mechanism for which a transparent driver interface exists (k The driver interface includes a fallback mechanism so that a driver can reject a request at runtime and let another driver handle the request. For each entry point, there must be at least three test runs with two or more drivers available with driver A configured to fall back to driver B, with one run where A returns `PSA_SUCCESS`, one where A returns `PSA_ERROR_NOT_SUPPORTED` and B is invoked, and one where A returns a different error and B is not invoked. -### Test framework +### Test drivers We have test drivers that are enabled by `PSA_CRYPTO_DRIVER_TEST` (not present in the usual config files, must be defined on the command line or in a custom @@ -145,8 +145,9 @@ transparent test driver. Each entry point is instrumented to record the number of hits for each part of the driver (same division as the files) and the status of the last call. It is -also possible to force the next call to return a specified status. See the -various `mbedtls_test_driver_XXX_hooks_t` structures declared by each driver. +also possible to force the next call to return a specified status, and +sometimes more things can be forced: see the various +`mbedtls_test_driver_XXX_hooks_t` structures declared by each driver. The drivers can use one of two back-ends: - internal: this requires the built-in implementation to be present. @@ -172,6 +173,12 @@ corresponding `mbedtls_psa_xxx()` function that it can call as its implementation - with the `libtestdriver1` back-end the function is called `libtestdriver1_mbedtls_psa_xxx()` instead. +A nice consequence of that strategy is that when an entry point has +test-driver support, most of the time, it automatically works for all +algorithms and key types supported by the library. (The exception being when +the driver needs to call a different function for different key types, as is +the case with some asymmetric key management operations.) + The renaming process for `libtestdriver1` is implemented as a few Perl regexes applied to a copy of the library code, see the `libtestdriver1.a` target in `tests/Makefile`. Another modification that's done to this copy is appending @@ -188,47 +195,220 @@ following sub-sections describe the status of the test driver support, mostly following the structure and order of sections 9.6 and 10.2 to 10.10 of the [PSA Crypto standard](https://arm-software.github.io/psa-api/crypto/1.1/) as that is also a natural division for implementing test drivers (that's how the -code is divided into files). It should be noted that the implementation -strategy ensures that when and entry point has test-driver support, it -automatically works for all algorithms and key types supported by the library, -thanks to the implementation strategy mentioned above. +code is divided into files). #### Key management -TODO +The following entry points are declared in `test/drivers/key_management.h`: + +- `"init"` (transparent and opaque) +- `"generate_key"` (transparent and opaque) +- `"export_public_key"` (transparent and opaque) +- `"import_key"` (transparent and opaque) +- `"export_key"` (opaque only) +- `"get_builtin_key"` (opaque only) +- `"copy_key"` (opaque only) + +The transparent driver fully implements the declared entry points, and can use +any backend: internal or libtestdriver1. + +The opaque's driver implementation status is as follows: +- `"generate_key"`: not implemented, always returns `NOT_SUPPORTED`. +- `"export_public_key"`: implemented only for ECC and RSA keys, both backends. +- `"import_key"`: implemented except for DH keys, both backends. +- `"export_key"`: implemented for built-in keys (ECC and AES), and for + non-builtin keys except DH keys. (Backend not relevant.) +- `"get_builtin_key"`: implemented - provisioned keys: AES-128 and ECC + secp2456r1. (Backend not relevant.) +- `"copy_key"`: implemented - emulates a SE without storage. (Backend not + relevant.) + +Note: the `"init"` entry point is not part of the "key management" family, but +listed here as it's declared and implemented in the same file. With the +transparent driver and the libtestdriver1 backend, it calls +`libtestdriver1_psa_crypto_init()`, which partially but not fully ensures +that this entry point is called before other entry points in the test drivers. +With the opaque driver, this entry point just does nothing an returns success. + +The following entry points are defined by the driver interface but missing +from our test drivers: +- `"allocate_key"`, `"destroy_key"`: this is for opaque drivers that store the + key material internally. + +Note: the instrumentation also allows forcing the output and its length. #### Message digests (Hashes) -TODO +The following entry points are declared (transparent only): +- `"hash_compute"` +- `"hash_setup"` +- `"hash_clone"` +- `"hash_update"` +- `"hash_finish"` +- `"hash_abort"` + +The transparent driver fully implements the declared entry points, and can use +any backend: internal or libtestdriver1. + +This familly is not part of the opaque driver as it doesn't use keys. #### Message authentication codes (MAC) -TODO +The following entry points are declared (transparent and opaque): +- `"mac_compute"` +- `"mac_sign_setup"` +- `"mac_verify_setup"` +- `"mac_update"` +- `"mac_sign_finish"` +- `"mac_verify_finish"` +- `"mac_abort"` + +The transparent driver fully implements the declared entry points, and can use +any backend: internal or libtestdriver1. + +The opaque driver only implements the instrumentation but not the actual +operations: entry points will always return `NOT_SUPPORTED`, unless another +status is forced. + +The following entry points are not implemented: +- `mac_verify`: this mostly makes sense for opaque drivers; the code will fall + back to using `"mac_compute"` if this is not implemented. So, perhaps +ideally we should test both with `"mac_verify"` implemented and with it not +implemented? Anyway, we have a test gap here. #### Unauthenticated ciphers -TODO +The following entry points are declared (transparent and opaque): +- `"cipher_encrypt"` +- `"cipher_decrypt"` +- `"cipher_encrypt_setup"` +- `"cipher_decrypt_setup"` +- `"cipher_set_iv"` +- `"cipher_update"` +- `"cipher_finish"` +- `"cipher_abort"` + +The transparent driver fully implements the declared entry points, and can use +any backend: internal or libtestdriver1. + +The opaque driver is not implemented at all, neither instumentation nor the +operation: entry points always return `NOT_SUPPORTED`. + +Note: the instrumentation also allows forcing a specific output and output +length. #### Authenticated encryption with associated data (AEAD) -TODO +The following entry points are declared (transparent only): +- `"aead_encrypt"` +- `"aead_decrypt"` +- `"aead_encrypt_setup"` +- `"aead_decrypt_setup"` +- `"aead_set_nonce"` +- `"aead_set_lengths"` +- `"aead_update_ad"` +- `"aead_update"` +- `"aead_finish"` +- `"aead_verify"` +- `"aead_abort"` + +The transparent driver fully implements the declared entry points, and can use +any backend: internal or libtestdriver1. + +The opaque driver does not implement or even declare entry points for this +family. + +Note: the instrumentation records the number of hits per entry point, not just +the total number of hits for this family. #### Key derivation -TODO +Not covered at all by the test drivers. + +That's a gap in our testing, as the driver interface does define a key +derivation family of entry points. This gap is probably related to the fact +that our internal code structure doesn't obey the guidelines and is not +aligned with the driver interface, see #5488 and related issues. #### Asymmetric signature -TODO +The following entry points are declared (transparent and opaque): + +- `"sign_message"` +- `"verify_message"` +- `"sign_hash"` +- `"verify_hash"` + +The transparent driver fully implements the declared entry points, and can use +any backend: internal or libtestdriver1. + +The opaque driver is not implemented at all, neither instumentation nor the +operation: entry points always return `NOT_SUPPORTED`. + +Note: the instrumentation also allows forcing a specific output and output +length, and has two instance of the hooks structure: one for sign, the other +for verify. + +Note: when a driver implements only the `"xxx_hash"` entry points, the core is +supposed to implement the `psa_xxx_message()` functions by computing the hash +itself before calling the `"xxx_hash"` entry point. Since the test driver does +implement the `"xxx_message"` entry point, it's not exercising that part of +the core's expected behaviour. #### Asymmetric encryption -TODO +The following entry points are declared (transparent and opaque): + +- `"asymmetric_encrypt"` +- `"asymmetric_decrypt"` + +The transparent driver fully implements the declared entry points, and can use +any backend: internal or libtestdriver1. + +The opaque driver is not implemented at all, neither instumentation nor the +operation: entry points always return `NOT_SUPPORTED`. + +Note: the instrumentation also allows forcing a specific output and output +length. #### Key agreement -TODO +The following entry points are declared (transparent and opaque): + +- `"key_agreement"` + +The transparent driver fully implements the declared entry points, and can use +any backend: internal or libtestdriver1. + +The opaque driver is not implemented at all, neither instumentation nor the +operation: entry points always return `NOT_SUPPORTED`. + +Note: the instrumentation also allows forcing a specific output and output +length. #### Other cryptographic services (Random number generation) -TODO +Not covered at all by the test drivers. + +The driver interface defines a `"get_entropy"` entry point, as well as a +"Random generation" family of entry points. None of those are currently +implemented in the library. Part of it will be planned for 4.0, see #8150. + +#### PAKE extension + +The following entry points are declared (transparent only): +- `"pake_setup"` +- `"pake_output"` +- `"pake_input"` +- `"pake_get_implicit_key"` +- `"pake_abort"` + +Note: the instrumentation records hits per entry point and allows forcing the +output and its length, as well as forcing the status of setup independently +from the others. + +The transparent driver fully implements the declared entry points, and can use +any backend: internal or libtestdriver1. + +The opaque driver does not implement or even declare entry points for this +family. From 733a67bb9a232674a44dbb8f016611e8921276b1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Wed, 15 Nov 2023 12:32:17 +0100 Subject: [PATCH 03/20] all.sh: group helper functions in sections MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- tests/scripts/all.sh | 62 +++++++++++++++++++++++--------------------- 1 file changed, 33 insertions(+), 29 deletions(-) diff --git a/tests/scripts/all.sh b/tests/scripts/all.sh index 3aabec41d4..698841b4b4 100755 --- a/tests/scripts/all.sh +++ b/tests/scripts/all.sh @@ -912,6 +912,39 @@ helper_libtestdriver1_adjust_config() { fi } +# Build the drivers library libtestdriver1.a (with ASan). +# +# Parameters: +# 1. a space-separated list of things to accelerate; +# 2. optional: a space-separate list of things to also support. +# Here "things" are PSA_WANT_ symbols but with PSA_WANT_ removed. +helper_libtestdriver1_make_drivers() { + loc_accel_flags=$( echo "$1 ${2-}" | sed 's/[^ ]* */-DLIBTESTDRIVER1_MBEDTLS_PSA_ACCEL_&/g' ) + make CC=$ASAN_CC -C tests libtestdriver1.a CFLAGS=" $ASAN_CFLAGS $loc_accel_flags" LDFLAGS="$ASAN_CFLAGS" +} + +# Build the main libraries, programs and tests, +# linking to the drivers library (with ASan). +# +# Parameters: +# 1. a space-separated list of things to accelerate; +# *. remaining arguments if any are passed directly to make +# (examples: lib, -C tests test_suite_xxx, etc.) +# Here "things" are PSA_WANT_ symbols but with PSA_WANT_ removed. +helper_libtestdriver1_make_main() { + loc_accel_list=$1 + shift + + # we need flags both with and without the LIBTESTDRIVER1_ prefix + loc_accel_flags=$( echo "$loc_accel_list" | sed 's/[^ ]* */-DLIBTESTDRIVER1_MBEDTLS_PSA_ACCEL_&/g' ) + loc_accel_flags="$loc_accel_flags $( echo "$loc_accel_list" | sed 's/[^ ]* */-DMBEDTLS_PSA_ACCEL_&/g' )" + make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -I../tests/include -I../tests -I../../tests -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_TEST_LIBTESTDRIVER1 $loc_accel_flags" LDFLAGS="-ltestdriver1 $ASAN_CFLAGS" "$@" +} + +################################################################ +#### Configuration helpers +################################################################ + # When called with no parameter this function disables all builtin curves. # The function optionally accepts 1 parameter: a space-separated list of the # curves that should be kept enabled. @@ -965,35 +998,6 @@ helper_get_psa_key_type_list() { echo "$loc_list" } -# Build the drivers library libtestdriver1.a (with ASan). -# -# Parameters: -# 1. a space-separated list of things to accelerate; -# 2. optional: a space-separate list of things to also support. -# Here "things" are PSA_WANT_ symbols but with PSA_WANT_ removed. -helper_libtestdriver1_make_drivers() { - loc_accel_flags=$( echo "$1 ${2-}" | sed 's/[^ ]* */-DLIBTESTDRIVER1_MBEDTLS_PSA_ACCEL_&/g' ) - make CC=$ASAN_CC -C tests libtestdriver1.a CFLAGS=" $ASAN_CFLAGS $loc_accel_flags" LDFLAGS="$ASAN_CFLAGS" -} - -# Build the main libraries, programs and tests, -# linking to the drivers library (with ASan). -# -# Parameters: -# 1. a space-separated list of things to accelerate; -# *. remaining arguments if any are passed directly to make -# (examples: lib, -C tests test_suite_xxx, etc.) -# Here "things" are PSA_WANT_ symbols but with PSA_WANT_ removed. -helper_libtestdriver1_make_main() { - loc_accel_list=$1 - shift - - # we need flags both with and without the LIBTESTDRIVER1_ prefix - loc_accel_flags=$( echo "$loc_accel_list" | sed 's/[^ ]* */-DLIBTESTDRIVER1_MBEDTLS_PSA_ACCEL_&/g' ) - loc_accel_flags="$loc_accel_flags $( echo "$loc_accel_list" | sed 's/[^ ]* */-DMBEDTLS_PSA_ACCEL_&/g' )" - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -I../tests/include -I../tests -I../../tests -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_TEST_LIBTESTDRIVER1 $loc_accel_flags" LDFLAGS="-ltestdriver1 $ASAN_CFLAGS" "$@" -} - ################################################################ #### Basic checks ################################################################ From 3dbd236b133674c5ede52ad569b513dbbd4d12a4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Wed, 15 Nov 2023 12:32:49 +0100 Subject: [PATCH 04/20] Update user-config-for-tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Was missing several key types and algs. Also, list those that are not implemented, but comment them out, to make it clearer what's not implemented yet. Signed-off-by: Manuel Pégourié-Gonnard --- tests/configs/user-config-for-test.h | 56 ++++++++++++++++++++++++---- 1 file changed, 49 insertions(+), 7 deletions(-) diff --git a/tests/configs/user-config-for-test.h b/tests/configs/user-config-for-test.h index 639496be60..479a153067 100644 --- a/tests/configs/user-config-for-test.h +++ b/tests/configs/user-config-for-test.h @@ -37,24 +37,61 @@ #endif /* Use the accelerator driver for all cryptographic mechanisms for which - * the test driver implemented. */ + * the test driver is implemented. This is copied from psa/crypto_config.h + * with the parts not implmented by the test driver commented out. */ +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_DERIVE +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_PASSWORD +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_PASSWORD_HASH +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_HMAC #define MBEDTLS_PSA_ACCEL_KEY_TYPE_AES +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_ARIA #define MBEDTLS_PSA_ACCEL_KEY_TYPE_CAMELLIA +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_CHACHA20 +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_DES #define MBEDTLS_PSA_ACCEL_KEY_TYPE_ECC_PUBLIC_KEY #define MBEDTLS_PSA_ACCEL_KEY_TYPE_ECC_KEY_PAIR_BASIC #define MBEDTLS_PSA_ACCEL_KEY_TYPE_ECC_KEY_PAIR_IMPORT #define MBEDTLS_PSA_ACCEL_KEY_TYPE_ECC_KEY_PAIR_EXPORT #define MBEDTLS_PSA_ACCEL_KEY_TYPE_ECC_KEY_PAIR_GENERATE -#define MBEDTLS_PSA_ACCEL_KEY_TYPE_RSA_KEY_PAIR +//#define MBEDTLS_PSA_ACCEL_KEY_TYPE_ECC_KEY_PAIR_DERIVE +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_DH_PUBLIC_KEY +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_DH_KEY_PAIR_BASIC +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_DH_KEY_PAIR_IMPORT +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_DH_KEY_PAIR_EXPORT +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_DH_KEY_PAIR_GENERATE +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_RAW_DATA +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_RSA_KEY_PAIR_BASIC +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_RSA_KEY_PAIR_IMPORT +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_RSA_KEY_PAIR_EXPORT +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_RSA_KEY_PAIR_GENERATE +#define MBEDTLS_PSA_ACCEL_KEY_TYPE_RSA_PUBLIC_KEY + #define MBEDTLS_PSA_ACCEL_ALG_CBC_NO_PADDING #define MBEDTLS_PSA_ACCEL_ALG_CBC_PKCS7 -#define MBEDTLS_PSA_ACCEL_ALG_CTR +#define MBEDTLS_PSA_ACCEL_ALG_CCM +#define MBEDTLS_PSA_ACCEL_ALG_CCM_STAR_NO_TAG +#define MBEDTLS_PSA_ACCEL_ALG_CMAC #define MBEDTLS_PSA_ACCEL_ALG_CFB -#define MBEDTLS_PSA_ACCEL_ALG_ECDSA +#define MBEDTLS_PSA_ACCEL_ALG_CHACHA20_POLY1305 +#define MBEDTLS_PSA_ACCEL_ALG_CTR #define MBEDTLS_PSA_ACCEL_ALG_DETERMINISTIC_ECDSA +#define MBEDTLS_PSA_ACCEL_ALG_ECB_NO_PADDING +#define MBEDTLS_PSA_ACCEL_ALG_ECDH +#define MBEDTLS_PSA_ACCEL_ALG_FFDH +#define MBEDTLS_PSA_ACCEL_ALG_ECDSA +#define MBEDTLS_PSA_ACCEL_ALG_JPAKE +#define MBEDTLS_PSA_ACCEL_ALG_GCM +//#define MBEDTLS_PSA_ACCEL_ALG_HKDF +//#define MBEDTLS_PSA_ACCEL_ALG_HKDF_EXTRACT +//#define MBEDTLS_PSA_ACCEL_ALG_HKDF_EXPAND +#define MBEDTLS_PSA_ACCEL_ALG_HMAC #define MBEDTLS_PSA_ACCEL_ALG_MD5 #define MBEDTLS_PSA_ACCEL_ALG_OFB +//#define MBEDTLS_PSA_ACCEL_ALG_PBKDF2_HMAC +//#define MBEDTLS_PSA_ACCEL_ALG_PBKDF2_AES_CMAC_PRF_128 #define MBEDTLS_PSA_ACCEL_ALG_RIPEMD160 +#define MBEDTLS_PSA_ACCEL_ALG_RSA_OAEP +#define MBEDTLS_PSA_ACCEL_ALG_RSA_PKCS1V15_CRYPT #define MBEDTLS_PSA_ACCEL_ALG_RSA_PKCS1V15_SIGN #define MBEDTLS_PSA_ACCEL_ALG_RSA_PSS #define MBEDTLS_PSA_ACCEL_ALG_SHA_1 @@ -62,9 +99,14 @@ #define MBEDTLS_PSA_ACCEL_ALG_SHA_256 #define MBEDTLS_PSA_ACCEL_ALG_SHA_384 #define MBEDTLS_PSA_ACCEL_ALG_SHA_512 -#define MBEDTLS_PSA_ACCEL_ALG_XTS -#define MBEDTLS_PSA_ACCEL_ALG_CMAC -#define MBEDTLS_PSA_ACCEL_ALG_HMAC +#define MBEDTLS_PSA_ACCEL_ALG_SHA3_224 +#define MBEDTLS_PSA_ACCEL_ALG_SHA3_256 +#define MBEDTLS_PSA_ACCEL_ALG_SHA3_384 +#define MBEDTLS_PSA_ACCEL_ALG_SHA3_512 +#define MBEDTLS_PSA_ACCEL_ALG_STREAM_CIPHER +//#define MBEDTLS_PSA_ACCEL_ALG_TLS12_PRF +//#define MBEDTLS_PSA_ACCEL_ALG_TLS12_PSK_TO_MS +//#define MBEDTLS_PSA_ACCEL_ALG_TLS12_ECJPAKE_TO_PMS #endif /* PSA_CRYPTO_DRIVER_TEST_ALL */ From 6a96f42051861d60ce8639377ca35d1d5f9a644d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Thu, 16 Nov 2023 13:01:22 +0100 Subject: [PATCH 05/20] Document driver wrapper suite & tested configs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The coverage data for the test drivers was generated using the following patch: diff --git a/scripts/lcov.sh b/scripts/lcov.sh index 9258ba788874..1ef071a65c06 100755 --- a/scripts/lcov.sh +++ b/scripts/lcov.sh @@ -63,8 +63,8 @@ if [ $# -gt 0 ] && [ "$1" = "--help" ]; then fi if in_mbedtls_build_dir; then - library_dir='library' - title='Mbed TLS' + library_dir='tests/src/drivers' + title='Mbed TLS test drivers' else library_dir='core' title='TF-PSA-Crypto' diff --git a/tests/scripts/all.sh b/tests/scripts/all.sh index 734d8323ca73..f6b17ca5692b 100755 --- a/tests/scripts/all.sh +++ b/tests/scripts/all.sh @@ -4795,14 +4795,17 @@ component_test_psa_crypto_drivers () { msg "build: full + test drivers dispatching to builtins" scripts/config.py full scripts/config.py unset MBEDTLS_PSA_CRYPTO_CONFIG - loc_cflags="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST_ALL" + loc_cflags="--coverage -DPSA_CRYPTO_DRIVER_TEST_ALL" loc_cflags="${loc_cflags} '-DMBEDTLS_USER_CONFIG_FILE=\"../tests/configs/user-config-for-test.h\"'" - loc_cflags="${loc_cflags} -I../tests/include -O2" + loc_cflags="${loc_cflags} -I../tests/include -Og -g3" - make CC=gcc CFLAGS="${loc_cflags}" LDFLAGS="$ASAN_CFLAGS" + make CC=gcc CFLAGS="${loc_cflags}" LDFLAGS="--coverage" -C tests test_suite_psa_crypto_driver_wrappers msg "test: full + test drivers dispatching to builtins" - make test + (cd tests && ./test_suite_psa_crypto_driver_wrappers --verbose) + #make test + + scripts/lcov.sh } component_test_make_shared () { Signed-off-by: Manuel Pégourié-Gonnard --- .../testing/driver-interface-test-strategy.md | 149 ++++++++++++++++++ 1 file changed, 149 insertions(+) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index 7c43a4feba..2551fd6735 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -412,3 +412,152 @@ any backend: internal or libtestdriver1. The opaque driver does not implement or even declare entry points for this family. + +### Driver wrapper test suite + +We have a test suite dedicated to driver dispatch, which takes advantage of the +instrumentation in the test drivers described in the previous section, in +order to check that drivers are called when they're supposed to, and that the +core behaves as expected when they return errors (in particular, that we fall +back to the built-in implementation when the driver returns `NOT_SUPPORTED). + +This is `test_suite_psa_crypto_driver_wrappers`, which is maintained manually +(that is, the test cases in the `.data` files are not auto-generated). The +entire test suite depends on the test drivers being enabled +(`PSA_CRYPTO_DRIVER_TEST`), which is not the case in the default or full +config. + +#### Configurations coverage + +The driver wrappers test suite has cases that expect both the driver and the +built-in to be present, and also cases that expect the driver to be present +but not the built-in. As such, it's impossible for a single configuration to +run all test cases, and we need at least two: driver+built-in, and +driver-only. + +- The driver+built-in case is covered by `test_psa_crypto_drivers` in `all.sh`. +This covers all areas (key types and algs) at once. +- The driver-only case is split into multiple `all.sh` components whose names + start with `test_psa_crypto_config_accel`; we have one or more component per +area, see below. + +Here's a summary of driver-only coverage, grouped by families of key types. + +Hash (key types: none) +- `test_psa_crypto_config_accel_hash`: all algs, default config, no parity + testing. +- `test_psa_crypto_config_accel_hash_use_psa`: all algs, full config, with + parity testing. + +HMAC (key type: HMAC) +- No driver-only testing here, see #8564. + +Cipher, AEAD and CMAC (key types: DES, AES, ARIA, CHACHA20, CAMELLIA): +- `test_psa_crypto_config_accel_cipher_aead`: all key types and algs, full + config with a few exclusions (PKCS5, PKCS12, NIST-KW), with parity testing. +- `test_psa_crypto_config_accel_cipher`: only DES (with all algs), full + config, no parity testing. +- `test_psa_crypto_config_accel_aead`: only AEAD algs (with all relevant key + types), full config, no parity testing. + +Key derivation (key types: `DERIVE`, `RAW_DATA`, `PASSWORD`, `PEPPER`, +`PASSWORD_HASH`): +- No testing as we don't have test driver support yet (see previous section). + +RSA (key types: `RSA_KEY_PAIR_xxx`, `RSA_PUBLIC_KEY`): +- `test_psa_crypto_config_accel_rsa_signature`: only signature algorithms, + default config, no parity testing. +- No testing of driver-only encryption yet, see #8553. + +DH (key types: `DH_KEY_PAIR_xxx`, `DH_PUBLIC_KEY`): +- `test_psa_crypto_config_accel_ffdh`: all key types and algs, full config, + with parity testing. +- `test_psa_crypto_config_accel_ecc_ffdh_no_bignum`: with also bignum removed. + +ECC (key types: `ECC_KEY_PAIR_xxx`, `ECC_PUBLIC_KEY`): +- Single algorithm accelerated (both key types, all curves): + - `test_psa_crypto_config_accel_ecdh`: default config, no parity testing. + - `test_psa_crypto_config_accel_ecdsa`: default config, no parity testing. + - `test_psa_crypto_config_accel_pake`: full config, no parity testing. +- All key types, algs and curves accelerated (full config with exceptions, + with parity testing): + - `test_psa_crypto_config_accel_ecc_ecp_light_only`: `ECP_C` mostly disabled + - `test_psa_crypto_config_accel_ecc_no_ecp_at_all`: `ECP_C` fully disabled + - `test_psa_crypto_config_accel_ecc_no_bignum`: `BIGNUM_C` disabled (DH disabled) + - `test_psa_crypto_config_accel_ecc_ffdh_no_bignum`: `BIGNUM_C` disabled (DH accelerated) +- Other - all algs accelerated but only some algs/curves (full config with + exceptions, no parity testing): + - `test_psa_crypto_config_accel_ecc_some_key_types` + - `test_psa_crypto_config_accel_ecc_non_weierstrass_curves` + - `test_psa_crypto_config_accel_ecc_weierstrass_curves` + +Note: `analyze_outcomes.py` provides a list of test cases that are not +executed in any configuration tested on the CI. Currently it flags some RSA +"fallback not available" tests, which is consistent with the fact that we're +missing testing driver-only RSA-encrypt testing. However, we're also missing +driver-only HMAC testing, but no test is flagged as never executed there; this +reveals we don't have "fallback not available" cases for MAC, see #8565. + +#### Test case coverage + +Since `test_suite_psa_crypto_driver_wrappers.data` is maintained manually, +we need to make sure it exercises all the cases that need to be tested. + +One way to evaluate this is to look at line coverage in test driver +implementaitons - this doesn't reveal all gaps, but it does reveal cases where +we thought about something when writing the test driver, but not when writing +test functions/data. + +Key management: +- `mbedtls_test_opaque_unwrap_key()` is never called. +- `mbedtls_test_transparent_generate_key()` is not tested with RSA keys. +- `mbedtls_test_transparent_import_key()` is not tested with DH keys. +- `mbedtls_test_opaque_import_key()` is not tested with unstructured keys nor + with RSA keys (nor DH keys since that's not implemented). +- `mbedtls_test_opaque_export_key()` is not tested with non-built-in keys. +- `mbedtls_test_transparent_export_public_key()` is not tested with RSA or DH keys. +- `mbedtls_test_opaque_export_public_key()` is not tested with non-built-in keys. +- `mbedtls_test_opaque_copy_key()` is not tested at all. + +Hash: +- `mbedtls_test_transparent_hash_finish()` is not tested with a forced status. + +MAC: +- The following are not tested with a forced status: + - `mbedtls_test_transparent_mac_sign_setup()` + - `mbedtls_test_transparent_mac_verify_setup()` + - `mbedtls_test_transparent_mac_update()` + - `mbedtls_test_transparent_mac_verify_finish()` + - `mbedtls_test_transparent_mac_abort()` +- No opaque entry point is tested (they're not implemented either). + +Cipher: +- The following are not tested with a forced status nor with a forced output: + - `mbedtls_test_transparent_cipher_encrypt()` + - `mbedtls_test_transparent_cipher_finish()` +- No opaque entry point is tested (they're not implemented either). + +AEAD: +- The following are not tested with a forced status: + - `mbedtls_test_transparent_aead_set_nonce()` + - `mbedtls_test_transparent_aead_set_lengths()` + - `mbedtls_test_transparent_aead_update_ad()` + - `mbedtls_test_transparent_aead_update()` + - `mbedtls_test_transparent_aead_finish()` + - `mbedtls_test_transparent_aead_verify()` +- `mbedtls_test_transparent_aead_verify()` is not tested with an invalid tag + (though it might be in another test suite). + +Signature: +- `sign_hash()` is not tested with RSA-PSS +- No opaque entry point is tested (they're not implemented either). + +Asymmetric encryption: +- No opaque entry point is tested (they're not implemented either). + +Key agreement: +- `mbedtls_test_transparent_key_agreement()` is not tested with FFDH. +- No opaque entry point is tested (they're not implemented either). + +PAKE: +- All lines are covered. From 1ad29c818be8ba78bfffee2ff32d2c602034e940 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Fri, 24 Nov 2023 11:30:28 +0100 Subject: [PATCH 06/20] Rm redundant driver+built-in all.sh component MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit As the comment says, this component's only goal was to make sure the legacy+driver test cases in test_suite_md.psa were executed. But actually these are already executed in component_test_psa_crypto_drivers which tests with everything having both a driver and the built-in, as can be seen in the outcomes file. Signed-off-by: Manuel Pégourié-Gonnard --- tests/scripts/all.sh | 20 -------------------- 1 file changed, 20 deletions(-) diff --git a/tests/scripts/all.sh b/tests/scripts/all.sh index 698841b4b4..104adb135b 100755 --- a/tests/scripts/all.sh +++ b/tests/scripts/all.sh @@ -3705,26 +3705,6 @@ component_test_psa_crypto_config_accel_hash () { make test } -component_test_psa_crypto_config_accel_hash_keep_builtins () { - msg "test: MBEDTLS_PSA_CRYPTO_CONFIG with accelerated+builtin hash" - # This component ensures that all the test cases for - # md_psa_dynamic_dispatch with legacy+driver in test_suite_md are run. - - loc_accel_list="ALG_MD5 ALG_RIPEMD160 ALG_SHA_1 \ - ALG_SHA_224 ALG_SHA_256 ALG_SHA_384 ALG_SHA_512 \ - ALG_SHA3_224 ALG_SHA3_256 ALG_SHA3_384 ALG_SHA3_512" - - # Start from default config (no USE_PSA) - helper_libtestdriver1_adjust_config "default" - - helper_libtestdriver1_make_drivers "$loc_accel_list" - - helper_libtestdriver1_make_main "$loc_accel_list" - - msg "test: MBEDTLS_PSA_CRYPTO_CONFIG with accelerated+builtin hash" - make test -} - # Auxiliary function to build config for hashes with and without drivers config_psa_crypto_hash_use_psa () { driver_only="$1" From 1f4c9051cd1726ae2cea20cacb1ddb8cbc3e7d55 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Fri, 24 Nov 2023 11:36:44 +0100 Subject: [PATCH 07/20] all.s: Rm redundant build-only accel components MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Most of them (2 exceptions, see below) are of the "driver + built-in" type, so they're all a subset of test_psa_crypto_driver which tests everything with driver + built-in at once. Furthermore, all those components were build-only, while test_psa_crypto_driver runs the test suites. Special cases: two of the components looked like they were trying to go for driver-only (ecdh disabling ECDH_C and hkdf disabling HKDF_C). For ECDH, built-in would actually be re-enabled because not enough was accelerated: you also need ECC key types and curves - see component_test_psa_crypto_config_accel_ecdh which does this correctly. For HKDF, we don't have test driver support for key derivation yet. I guess that shows how little testing value these build-only components really had. Signed-off-by: Manuel Pégourié-Gonnard --- tests/scripts/all.sh | 248 ------------------------------------------- 1 file changed, 248 deletions(-) diff --git a/tests/scripts/all.sh b/tests/scripts/all.sh index 104adb135b..a49d1c65fe 100755 --- a/tests/scripts/all.sh +++ b/tests/scripts/all.sh @@ -4158,254 +4158,6 @@ component_test_ccm_aes_sha256() { make test } -# This should be renamed to test and updated once the accelerator ECDH code is in place and ready to test. -component_build_psa_accel_alg_ecdh() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_ECDH without MBEDTLS_ECDH_C" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py unset MBEDTLS_ECDH_C - scripts/config.py unset MBEDTLS_KEY_EXCHANGE_ECDH_RSA_ENABLED - scripts/config.py unset MBEDTLS_KEY_EXCHANGE_ECDH_ECDSA_ENABLED - scripts/config.py unset MBEDTLS_KEY_EXCHANGE_ECDHE_ECDSA_ENABLED - scripts/config.py unset MBEDTLS_KEY_EXCHANGE_ECDHE_RSA_ENABLED - scripts/config.py unset MBEDTLS_KEY_EXCHANGE_ECDHE_PSK_ENABLED - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_ECDH -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator HMAC code is in place and ready to test. -component_build_psa_accel_alg_hmac() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_HMAC" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_HMAC -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator HKDF code is in place and ready to test. -component_build_psa_accel_alg_hkdf() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_HKDF without MBEDTLS_HKDF_C" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_HKDF_C - # Make sure to unset TLS1_3 since it requires HKDF_C and will not build properly without it. - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_HKDF -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator MD5 code is in place and ready to test. -component_build_psa_accel_alg_md5() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_MD5 - other hashes" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RIPEMD160 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_1 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_224 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_256 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_384 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_512 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_TLS12_ECJPAKE_TO_PMS - scripts/config.py unset MBEDTLS_LMS_C - scripts/config.py unset MBEDTLS_LMS_PRIVATE - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_MD5 -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator RIPEMD160 code is in place and ready to test. -component_build_psa_accel_alg_ripemd160() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_RIPEMD160 - other hashes" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_MD5 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_1 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_224 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_256 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_384 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_512 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_TLS12_ECJPAKE_TO_PMS - scripts/config.py unset MBEDTLS_LMS_C - scripts/config.py unset MBEDTLS_LMS_PRIVATE - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_RIPEMD160 -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator SHA1 code is in place and ready to test. -component_build_psa_accel_alg_sha1() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_SHA_1 - other hashes" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_MD5 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RIPEMD160 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_224 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_256 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_384 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_512 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_TLS12_ECJPAKE_TO_PMS - scripts/config.py unset MBEDTLS_LMS_C - scripts/config.py unset MBEDTLS_LMS_PRIVATE - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_SHA_1 -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator SHA224 code is in place and ready to test. -component_build_psa_accel_alg_sha224() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_SHA_224 - other hashes" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_MD5 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RIPEMD160 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_1 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_384 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_512 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_TLS12_ECJPAKE_TO_PMS - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_SHA_224 -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator SHA256 code is in place and ready to test. -component_build_psa_accel_alg_sha256() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_SHA_256 - other hashes" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_MD5 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RIPEMD160 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_1 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_224 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_384 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_512 - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_SHA_256 -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator SHA384 code is in place and ready to test. -component_build_psa_accel_alg_sha384() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_SHA_384 - other hashes" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_MD5 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RIPEMD160 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_1 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_224 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_256 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_TLS12_ECJPAKE_TO_PMS - scripts/config.py unset MBEDTLS_LMS_C - scripts/config.py unset MBEDTLS_LMS_PRIVATE - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_SHA_384 -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator SHA512 code is in place and ready to test. -component_build_psa_accel_alg_sha512() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_SHA_512 - other hashes" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_MD5 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RIPEMD160 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_1 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_224 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_256 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_SHA_384 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_TLS12_ECJPAKE_TO_PMS - scripts/config.py unset MBEDTLS_LMS_C - scripts/config.py unset MBEDTLS_LMS_PRIVATE - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_SHA_512 -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator RSA code is in place and ready to test. -component_build_psa_accel_alg_rsa_pkcs1v15_crypt() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_RSA_PKCS1V15_CRYPT + PSA_WANT_KEY_TYPE_RSA_PUBLIC_KEY" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_ALG_RSA_PKCS1V15_CRYPT 1 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_PKCS1V15_SIGN - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_OAEP - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_PSS - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_RSA_PKCS1V15_CRYPT -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator RSA code is in place and ready to test. -component_build_psa_accel_alg_rsa_pkcs1v15_sign() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_RSA_PKCS1V15_SIGN + PSA_WANT_KEY_TYPE_RSA_PUBLIC_KEY" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_ALG_RSA_PKCS1V15_SIGN 1 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_PKCS1V15_CRYPT - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_OAEP - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_PSS - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_RSA_PKCS1V15_SIGN -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator RSA code is in place and ready to test. -component_build_psa_accel_alg_rsa_oaep() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_RSA_OAEP + PSA_WANT_KEY_TYPE_RSA_PUBLIC_KEY" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_ALG_RSA_OAEP 1 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_PKCS1V15_CRYPT - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_PKCS1V15_SIGN - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_PSS - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_RSA_OAEP -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator RSA code is in place and ready to test. -component_build_psa_accel_alg_rsa_pss() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_ALG_RSA_PSS + PSA_WANT_KEY_TYPE_RSA_PUBLIC_KEY" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_ALG_RSA_PSS 1 - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_PKCS1V15_CRYPT - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_PKCS1V15_SIGN - scripts/config.py -f "$CRYPTO_CONFIG_H" unset PSA_WANT_ALG_RSA_OAEP - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_ALG_RSA_PSS -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator RSA code is in place and ready to test. -component_build_psa_accel_key_type_rsa_key_pair() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_KEY_TYPE_RSA_KEY_PAIR_xxx + PSA_WANT_ALG_RSA_PSS" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_ALG_RSA_PSS 1 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_KEY_TYPE_RSA_KEY_PAIR_BASIC 1 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_KEY_TYPE_RSA_KEY_PAIR_IMPORT 1 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_KEY_TYPE_RSA_KEY_PAIR_EXPORT 1 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_KEY_TYPE_RSA_KEY_PAIR_GENERATE 1 - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_KEY_TYPE_RSA_KEY_PAIR -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - -# This should be renamed to test and updated once the accelerator RSA code is in place and ready to test. -component_build_psa_accel_key_type_rsa_public_key() { - msg "build: full - MBEDTLS_USE_PSA_CRYPTO + PSA_WANT_KEY_TYPE_RSA_PUBLIC_KEY + PSA_WANT_ALG_RSA_PSS" - scripts/config.py full - scripts/config.py unset MBEDTLS_USE_PSA_CRYPTO - scripts/config.py unset MBEDTLS_SSL_PROTO_TLS1_3 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_ALG_RSA_PSS 1 - scripts/config.py -f "$CRYPTO_CONFIG_H" set PSA_WANT_KEY_TYPE_RSA_PUBLIC_KEY 1 - # Need to define the correct symbol and include the test driver header path in order to build with the test driver - make CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_PSA_ACCEL_KEY_TYPE_RSA_PUBLIC_KEY -I../tests/include" LDFLAGS="$ASAN_CFLAGS" -} - - support_build_tfm_armcc () { support_build_armcc } From b18bc8013324aa3c1684518361d0edff1e6f626c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Fri, 24 Nov 2023 11:59:25 +0100 Subject: [PATCH 08/20] Add note about fallback to other entry points MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- .../testing/driver-interface-test-strategy.md | 21 ++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index 2551fd6735..d9a92d1ff7 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -165,6 +165,25 @@ libtestdriver1 too. - Performance might be better with internal though? - The instrumentation works the same with both back-ends. +Note: our test drivers tend to provide all possible entry points (with a few +exceptions that may not be intentional, see the next sections). However, in +some cases, when an entry point is not available, the core is supposed to +implement it using other entry points, for example: +- `mac_verify` may use `mac_compute` if the driver does no provide verify; +- for things that have both one-shot and multi-part API, the driver can + provide only the multi-part entry points, and the core is supposed to +implement one-shot on top of it (but still call the one-shot entry points when +they're available); +- `sign/verify_message` can be implemented on top of `sign/verify_hash` for + some algorithms; +- (not sure if the list is exhaustive). + +Ideally, we'd want build options for the test drivers so that we can test with +different combinations of entry points present, and make sure the core behaves +appropriately when some entry points are absent but other entry points allow +implementing the operation. This is currently not supported by our test +drivers. + Our implementation of PSA Crypto is structured in a way that the built-in implementation of each operation follows the driver API, see [`../architecture/psa-crypto-implementation-structure.md`](../architecture/psa-crypto-implementation-structure.html). @@ -271,7 +290,7 @@ operations: entry points will always return `NOT_SUPPORTED`, unless another status is forced. The following entry points are not implemented: -- `mac_verify`: this mostly makes sense for opaque drivers; the code will fall +- `mac_verify`: this mostly makes sense for opaque drivers; the core will fall back to using `"mac_compute"` if this is not implemented. So, perhaps ideally we should test both with `"mac_verify"` implemented and with it not implemented? Anyway, we have a test gap here. From 4c81c343acc6214b995281f348f5737ae28eb3e7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Fri, 24 Nov 2023 12:00:15 +0100 Subject: [PATCH 09/20] Fix copy-pasta in top-of-file comment MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- tests/src/drivers/test_driver_pake.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/src/drivers/test_driver_pake.c b/tests/src/drivers/test_driver_pake.c index a0b6c1cb0c..52395e4d0e 100644 --- a/tests/src/drivers/test_driver_pake.c +++ b/tests/src/drivers/test_driver_pake.c @@ -1,5 +1,5 @@ /* - * Test driver for MAC entry points. + * Test driver for PAKE entry points. */ /* Copyright The Mbed TLS Contributors * SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later From 70cd911405d576456004aae89950033a703c9c6b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Fri, 24 Nov 2023 12:06:48 +0100 Subject: [PATCH 10/20] Improve comment in a header file MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- .../crypto_config_test_driver_extension.h | 25 +++++++++++++++---- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/tests/include/test/drivers/crypto_config_test_driver_extension.h b/tests/include/test/drivers/crypto_config_test_driver_extension.h index dac07acd33..66378e7def 100644 --- a/tests/include/test/drivers/crypto_config_test_driver_extension.h +++ b/tests/include/test/drivers/crypto_config_test_driver_extension.h @@ -1,9 +1,24 @@ /** - * This file is intended to be used to build PSA test driver libraries. It is - * intended to be appended by the test build system to the crypto_config.h file - * of the Mbed TLS library the test library will be linked to. It mirrors the - * PSA_ACCEL_* macros defining the cryptographic operations the test library - * supports. + * This file is intended to be used to build PSA external test driver + * libraries (libtestdriver1). + * + * It is intended to be appended by the test build system to the + * crypto_config.h file of the Mbed TLS library the test library will be + * linked to (see `tests/Makefile` libtestdriver1 target). This is done in + * order to insert it at the right time: after the main configuration + * (PSA_WANT) but before the logic that determines what built-ins to enable + * based on PSA_WANT and MBEDTLS_PSA_ACCEL macros. + * + * It reverses the PSA_ACCEL_* macros defining the cryptographic operations + * that will be accelerated in the main library: + * - When something is accelerated in the main library, we need it supported + * in libtestdriver1, so we disable the accel macro in order to the built-in + * to be enabled. + * - When something is NOT accelerated in the main library, we don't need it + * in libtestdriver1, so we enable its accel macro in order to the built-in + * to be disabled, to keep libtestdriver1 minimal. (We can't adjust the + * PSA_WANT macros as they need to be the same between libtestdriver1 and + * the main library, since they determine the ABI between the two.) */ #include "psa/crypto_legacy.h" From f2089dab5ed8d742d9115aa4d9cb696ce4c00044 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Mon, 18 Dec 2023 11:36:26 +0100 Subject: [PATCH 11/20] Update status of RSA testing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Improved by https://github.com/Mbed-TLS/mbedtls/pull/8616/ - closing 8553. Signed-off-by: Manuel Pégourié-Gonnard --- .../testing/driver-interface-test-strategy.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index d9a92d1ff7..73200aee9d 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -484,9 +484,9 @@ Key derivation (key types: `DERIVE`, `RAW_DATA`, `PASSWORD`, `PEPPER`, - No testing as we don't have test driver support yet (see previous section). RSA (key types: `RSA_KEY_PAIR_xxx`, `RSA_PUBLIC_KEY`): -- `test_psa_crypto_config_accel_rsa_signature`: only signature algorithms, - default config, no parity testing. -- No testing of driver-only encryption yet, see #8553. +- `test_psa_crypto_config_accel_rsa_crypto`: all 4 algs (encryption & + signature, v1.5 & v2.1), config `crypto_full`, with parity testing excluding +PK. DH (key types: `DH_KEY_PAIR_xxx`, `DH_PUBLIC_KEY`): - `test_psa_crypto_config_accel_ffdh`: all key types and algs, full config, @@ -511,11 +511,9 @@ ECC (key types: `ECC_KEY_PAIR_xxx`, `ECC_PUBLIC_KEY`): - `test_psa_crypto_config_accel_ecc_weierstrass_curves` Note: `analyze_outcomes.py` provides a list of test cases that are not -executed in any configuration tested on the CI. Currently it flags some RSA -"fallback not available" tests, which is consistent with the fact that we're -missing testing driver-only RSA-encrypt testing. However, we're also missing -driver-only HMAC testing, but no test is flagged as never executed there; this -reveals we don't have "fallback not available" cases for MAC, see #8565. +executed in any configuration tested on the CI. We're missing driver-only HMAC +testing, but no test is flagged as never executed there; this reveals we don't +have "fallback not available" cases for MAC, see #8565. #### Test case coverage From 45fe86db99fc9d453059981df32a1c94aed1bcc5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Wed, 20 Dec 2023 12:43:13 +0100 Subject: [PATCH 12/20] Fix a typo in a comment MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- tests/configs/user-config-for-test.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/configs/user-config-for-test.h b/tests/configs/user-config-for-test.h index 479a153067..f40f83895f 100644 --- a/tests/configs/user-config-for-test.h +++ b/tests/configs/user-config-for-test.h @@ -38,7 +38,7 @@ /* Use the accelerator driver for all cryptographic mechanisms for which * the test driver is implemented. This is copied from psa/crypto_config.h - * with the parts not implmented by the test driver commented out. */ + * with the parts not implemented by the test driver commented out. */ #define MBEDTLS_PSA_ACCEL_KEY_TYPE_DERIVE #define MBEDTLS_PSA_ACCEL_KEY_TYPE_PASSWORD #define MBEDTLS_PSA_ACCEL_KEY_TYPE_PASSWORD_HASH From 98f8da1b1ad1b3a4291634a60be5f3ddfc8d3609 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Wed, 10 Jan 2024 12:53:58 +0100 Subject: [PATCH 13/20] Update names of components renamed in the meantime MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- docs/architecture/testing/driver-interface-test-strategy.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index 73200aee9d..5bf1375f32 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -472,9 +472,9 @@ HMAC (key type: HMAC) - No driver-only testing here, see #8564. Cipher, AEAD and CMAC (key types: DES, AES, ARIA, CHACHA20, CAMELLIA): -- `test_psa_crypto_config_accel_cipher_aead`: all key types and algs, full - config with a few exclusions (PKCS5, PKCS12, NIST-KW), with parity testing. -- `test_psa_crypto_config_accel_cipher`: only DES (with all algs), full +- `test_psa_crypto_config_accel_cipher_aead_cmac`: all key types and algs, full + config with a few exclusions (NIST-KW), with parity testing. +- `test_psa_crypto_config_accel_des`: only DES (with all algs), full config, no parity testing. - `test_psa_crypto_config_accel_aead`: only AEAD algs (with all relevant key types), full config, no parity testing. From 6c45361a9c8f927b25e2448bbb5e66ee85b6d9e0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Mon, 18 Mar 2024 10:12:49 +0100 Subject: [PATCH 14/20] Update for HMAC testing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Been merged in the meantime. Signed-off-by: Manuel Pégourié-Gonnard --- docs/architecture/testing/driver-interface-test-strategy.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index 5bf1375f32..f4f224b85e 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -469,7 +469,9 @@ Hash (key types: none) parity testing. HMAC (key type: HMAC) -- No driver-only testing here, see #8564. +- `test_psa_crypto_config_accel_hmac`: all algs, full config except a few + exclusions (PKCS5, PKCS7, HMAC-DRBG, legacy HKDF, deterministic ECDSA), with +parity testing. Cipher, AEAD and CMAC (key types: DES, AES, ARIA, CHACHA20, CAMELLIA): - `test_psa_crypto_config_accel_cipher_aead_cmac`: all key types and algs, full From dde1abd5724edd5ef5bd915a3d191b69a3ab25ba Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Tue, 9 Apr 2024 12:12:48 +0200 Subject: [PATCH 15/20] Update of opaque asymmetric encrypt/decrypt MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit https://github.com/Mbed-TLS/mbedtls/pull/8700 merged in the meantime. Signed-off-by: Manuel Pégourié-Gonnard --- .../testing/driver-interface-test-strategy.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index f4f224b85e..dfec4b3781 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -384,8 +384,10 @@ The following entry points are declared (transparent and opaque): The transparent driver fully implements the declared entry points, and can use any backend: internal or libtestdriver1. -The opaque driver is not implemented at all, neither instumentation nor the -operation: entry points always return `NOT_SUPPORTED`. +The opaque driver implements the declared entry points, and can use any +backend: internal or libtestdriver1. However it does not implement the +instrumentation (hits, forced output/status), as this [was not an immediate +priority](https://github.com/Mbed-TLS/mbedtls/pull/8700#issuecomment-1892466159). Note: the instrumentation also allows forcing a specific output and output length. @@ -528,7 +530,6 @@ we thought about something when writing the test driver, but not when writing test functions/data. Key management: -- `mbedtls_test_opaque_unwrap_key()` is never called. - `mbedtls_test_transparent_generate_key()` is not tested with RSA keys. - `mbedtls_test_transparent_import_key()` is not tested with DH keys. - `mbedtls_test_opaque_import_key()` is not tested with unstructured keys nor @@ -571,9 +572,6 @@ Signature: - `sign_hash()` is not tested with RSA-PSS - No opaque entry point is tested (they're not implemented either). -Asymmetric encryption: -- No opaque entry point is tested (they're not implemented either). - Key agreement: - `mbedtls_test_transparent_key_agreement()` is not tested with FFDH. - No opaque entry point is tested (they're not implemented either). From 0ca2fd0e2b18ae353e996ce48bb367d45c27eee0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Fri, 12 Apr 2024 10:14:17 +0200 Subject: [PATCH 16/20] Update libtestdriver1 vs internal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- .../testing/driver-interface-test-strategy.md | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index dfec4b3781..89f3c9b843 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -155,15 +155,11 @@ The drivers can use one of two back-ends: the build. Historical note: internal was initially the only back-end; then support for -libtestdriver1 was added gradually. - -Question: if/when we have complete libtestdriver1 support, do we still need -internal? Thoughts: -- It's useful to have builds with both a driver and the built-in, in -order to test fallback to built-in, but this could be achieved with -libtestdriver1 too. - - Performance might be better with internal though? -- The instrumentation works the same with both back-ends. +libtestdriver1 was added gradually. Support for libtestdriver1 is now complete +(see following sub-sections), so we could remove internal now. Note it's +useful to have builds with both a driver and the built-in, in order to test +fallback to built-in, which is currently done only with internal, but this can +be achieved with libtestdriver1 just as well. Note: our test drivers tend to provide all possible entry points (with a few exceptions that may not be intentional, see the next sections). However, in From ae22f04769b754904a0b6fc188a54319381b702b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Fri, 12 Apr 2024 10:18:27 +0200 Subject: [PATCH 17/20] Refine paragraphs about incomplete entry points MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- docs/architecture/testing/driver-interface-test-strategy.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index 89f3c9b843..ecd13a5dc3 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -177,8 +177,10 @@ they're available); Ideally, we'd want build options for the test drivers so that we can test with different combinations of entry points present, and make sure the core behaves appropriately when some entry points are absent but other entry points allow -implementing the operation. This is currently not supported by our test -drivers. +implementing the operation. This will remain hard to test until we have proper +support for JSON-defined drivers with auto-generation of dispatch code. +(The `MBEDTLS_PSA_ACCEL_xxx` macros we currently use are not expressive enough +to specify which entry points are support for a given mechanism.) Our implementation of PSA Crypto is structured in a way that the built-in implementation of each operation follows the driver API, see From a47a3c4e13cab525cf075a5c83229460273ddc31 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Fri, 12 Apr 2024 10:21:42 +0200 Subject: [PATCH 18/20] Rephrase description of the KDF situation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- .../testing/driver-interface-test-strategy.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index ecd13a5dc3..3c0e068fdd 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -342,10 +342,9 @@ the total number of hits for this family. Not covered at all by the test drivers. -That's a gap in our testing, as the driver interface does define a key -derivation family of entry points. This gap is probably related to the fact -that our internal code structure doesn't obey the guidelines and is not -aligned with the driver interface, see #5488 and related issues. +That's a test gap which reflects a feature gap: the driver interface does +define a key derivation family of entry points, but we don't currently +implement that part of the driver interface, see #5488 and related issues. #### Asymmetric signature @@ -483,7 +482,7 @@ Cipher, AEAD and CMAC (key types: DES, AES, ARIA, CHACHA20, CAMELLIA): Key derivation (key types: `DERIVE`, `RAW_DATA`, `PASSWORD`, `PEPPER`, `PASSWORD_HASH`): -- No testing as we don't have test driver support yet (see previous section). +- No testing as we don't have driver support yet (see previous section). RSA (key types: `RSA_KEY_PAIR_xxx`, `RSA_PUBLIC_KEY`): - `test_psa_crypto_config_accel_rsa_crypto`: all 4 algs (encryption & From 432e3b41989c8b934a34bd209e2c67ed7baaf774 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Fri, 12 Apr 2024 10:25:25 +0200 Subject: [PATCH 19/20] Misc fixes & improvements to driver testing doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Manuel Pégourié-Gonnard --- .../testing/driver-interface-test-strategy.md | 24 +++++++++++++------ 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index 3c0e068fdd..e9ac02e1e9 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -194,7 +194,10 @@ A nice consequence of that strategy is that when an entry point has test-driver support, most of the time, it automatically works for all algorithms and key types supported by the library. (The exception being when the driver needs to call a different function for different key types, as is -the case with some asymmetric key management operations.) +the case with some asymmetric key management operations.) (Note: it's still +useful to test drivers in configurations with partial algorithm support, and +that can still be done by configuring libtestdriver1 and the main library as +desired.) The renaming process for `libtestdriver1` is implemented as a few Perl regexes applied to a copy of the library code, see the `libtestdriver1.a` target in @@ -437,7 +440,7 @@ We have a test suite dedicated to driver dispatch, which takes advantage of the instrumentation in the test drivers described in the previous section, in order to check that drivers are called when they're supposed to, and that the core behaves as expected when they return errors (in particular, that we fall -back to the built-in implementation when the driver returns `NOT_SUPPORTED). +back to the built-in implementation when the driver returns `NOT_SUPPORTED`). This is `test_suite_psa_crypto_driver_wrappers`, which is maintained manually (that is, the test cases in the `.data` files are not auto-generated). The @@ -445,6 +448,12 @@ entire test suite depends on the test drivers being enabled (`PSA_CRYPTO_DRIVER_TEST`), which is not the case in the default or full config. +The test suite is focused on driver usage (mostly by checking the expected +number of hits) but also does some validation of the results: for +deterministic algorithms, known-answers tests are used, and for the rest, some +consistency checks are done (more or less detailled depending on the algorithm +and build configuration). + #### Configurations coverage The driver wrappers test suite has cases that expect both the driver and the @@ -519,12 +528,13 @@ have "fallback not available" cases for MAC, see #8565. #### Test case coverage Since `test_suite_psa_crypto_driver_wrappers.data` is maintained manually, -we need to make sure it exercises all the cases that need to be tested. +we need to make sure it exercises all the cases that need to be tested. In the +future, this file should be generated in order to ensure exhaustiveness. -One way to evaluate this is to look at line coverage in test driver -implementaitons - this doesn't reveal all gaps, but it does reveal cases where -we thought about something when writing the test driver, but not when writing -test functions/data. +In the meantime, one way to observe (lack of) completeness is to look at line +coverage in test driver implementaitons - this doesn't reveal all gaps, but it +does reveal cases where we thought about something when writing the test +driver, but not when writing test functions/data. Key management: - `mbedtls_test_transparent_generate_key()` is not tested with RSA keys. From 4575d230bf76928d955827b93b3608f4ef9a70b7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20P=C3=A9gouri=C3=A9-Gonnard?= Date: Mon, 15 Apr 2024 10:54:49 +0200 Subject: [PATCH 20/20] Add a note on hits usefulness MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit And fix a typo while at it. Signed-off-by: Manuel Pégourié-Gonnard --- .../testing/driver-interface-test-strategy.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index e9ac02e1e9..5fc5e18e6d 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -147,7 +147,8 @@ Each entry point is instrumented to record the number of hits for each part of the driver (same division as the files) and the status of the last call. It is also possible to force the next call to return a specified status, and sometimes more things can be forced: see the various -`mbedtls_test_driver_XXX_hooks_t` structures declared by each driver. +`mbedtls_test_driver_XXX_hooks_t` structures declared by each driver (and +subsections below). The drivers can use one of two back-ends: - internal: this requires the built-in implementation to be present. @@ -161,6 +162,15 @@ useful to have builds with both a driver and the built-in, in order to test fallback to built-in, which is currently done only with internal, but this can be achieved with libtestdriver1 just as well. +Note on instrumentation: originally, when only the internal backend was +available, hits were how we knew that the driver was called, as opposed to +directly calling the built-in code. With libtestdriver1, we can check that by +ensuring that the built-in code is not present, so if the operation gives the +correct result, only a driver call can have calculated that result. So, +nowadays there is low value in checking the hit count. There is still some +value for hit counts, e.g. checking that we don't call a multipart entry point +when we intended to call the one-shot entry point, but it's limited. + Note: our test drivers tend to provide all possible entry points (with a few exceptions that may not be intentional, see the next sections). However, in some cases, when an entry point is not available, the core is supposed to @@ -180,7 +190,7 @@ appropriately when some entry points are absent but other entry points allow implementing the operation. This will remain hard to test until we have proper support for JSON-defined drivers with auto-generation of dispatch code. (The `MBEDTLS_PSA_ACCEL_xxx` macros we currently use are not expressive enough -to specify which entry points are support for a given mechanism.) +to specify which entry points are supported for a given mechanism.) Our implementation of PSA Crypto is structured in a way that the built-in implementation of each operation follows the driver API, see