ECDSA has two variants: deterministic (PSA_ALG_DETERMINISTIC_ECDSA) and
randomized (PSA_ALG_ECDSA). The two variants are different for signature but
identical for verification. Mbed TLS accepts either variant as the algorithm
parameter for verification even when only the other variant is supported,
so we need to handle this as a special case when generating not-supported
test cases.
In this commit:
* Automatically generated not-supported test cases for ECDSA now require
both variants to be disabled.
* Add manually written not-supported test cases for the signature
operation when exactly one variant is supported.
* Add manually written positive test cases for the verification
operation when exactly one variant is supported.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
If we don't exclude them from test case enumeration, then
detect_not_implemented_dependencies would cause the generated test cases to
be commented out, but the test case generation would fail before that
because asymmetric_key_data.py doesn't include DH and DSA keys.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When we generate a test case for a mechanism that is not implemented,
comment out the test case rather than giving it a never-fulfilled
dependency. That way we don't create test cases that cannot be executed.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Allow "skipping" a test case, meaning that the test case is generated
commented out. This is useful when systematically generating test cases
according to certain rules, where some generated tests cannot be executed
but we still want them to be visible when auditing the generation output.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Move hack_dependencies_not_implemented into a class to make the file
structure easier to understand and reduce the visibility of the
_implemented_dependencies cache. Rename it because it's no longer a
temporary hack (originally intended to work around the fact that not all
PSA_WANT symbols were implemented), it's now a way to detect test cases for
cryptographic mechanisms that are declared but not implemented.
Internal refactoring only. No behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In automatically generated PSA test cases, we detect cryptographic
mechanisms that are not implemented, and skip the corresponding test cases.
Originally this detection was intended for mechanisms for which the PSA_WANT
symbols were not implemented, but then it morphed into skipping mechanisms
that are declared in crypto_values.h but not actually implemented. So it no
longer makes sense to skip the test cases for which a negative
dependency (!PSA_WANT_xxx) is not implemented.
This causes more not-supported test cases to run.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use psa_information.TestCase for operation failure test cases.
This changes the generated output in two ways:
* Not-implemented mechanisms now have a `DEPENDENCY_NOT_IMPLEMENTED_YET_xxx`
dependency in addition to the never-fulfilled `PSA_WANT_xxx` dependency.
This does not affect when test cases run.
* ECC test cases now have correct dependency symbols, e.g.
`PSA_WANT_ECC_SECP_R1_192` instead of `PSA_WANT_ECC_FAMILY_SECP_R1`. This
is a bug fix: ECC test cases were formerly never executed because of
incorrect dependency symbols.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use psa_information.TestCase for not-supported test cases for key import and
generation.
No change to the generated output.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In automatically generated PSA test cases, annotate the test cases that are
expected to be never executed due to a dependency that is not implemented.
This was already done for not-supported test cases and for key generation,
but not for positive test cases of key usage.
You can audit which mechanisms are detected as not-implemented with
```
grep -hEo 'DEPENDENCY_NOT_IMPLEMENTED_YET_\w+' tests/suites/*.data | sort -u
```
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
To determine PSA mechanisms that are not implemented, also read PSA_WANT
symbols that cannot (or are not intended to) be configured independently,
and thus are not listed in psa/crypto_config.h. Find those symbols in
the config adjustment header mbedtls/config_psa.h.
No impact on generated files yet, because hack_dependencies_not_implemented
is currently only used on key types that have explicit dependencies. This
will allow using hack_dependencies_not_implemented in other places, for
example to handle algorithm variants like PSA_WANT_ALG_ECDSA_ANY which is
inferred from PSA_WANT_ALG_ECDSA.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Create a class for test cases with automatically inferred dependencies,
suitable for PSA crypto API test cases. As of this commit, only basic cases
that use a key are supported. Subsequent commits will address negative tests
and key generation tests that require fancier handling of dependencies.
No change to the generated output.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The output is less nice, since it no longer mostly matches the order in
which symbols are used in the test case arguments. But this makes the output
more canonical, so it will be easier to notice if semantic changes occur in
subsequent commits.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Indicate which dependencies are not implemented. This makes it feasible to
audit the not-implemented detection.
The generated test cases that are detected as never-executed now have one or
more `DEPENDENCY_NOT_IMPLEMENTED_YET_PSA_WANT_xxx` instead of a single
`DEPENDENCY_NOT_IMPLEMENTED`. This does not affect when the test cases run.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
secp224k1 is the one with 225-bit private keys.
The consequences of this mistake were:
* We emitted positive test cases for hypothetical SECP_R1_225 and
SECP_K1_224 curves, which were never executed.
* We emitted useless not-supported test cases for SECP_R1_225 and SECP_K1_224.
* We were missing positive test cases for SECP_R1_224 in automatically
generated tests.
* We were missing not-supported test cases for SECP_R1_224 and SECP_K1_225.
Thus this didn't cause test failures, but it caused missing test coverage
and some never-executed test cases.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Fix // comments stopping on 'n' instead of newlines. Also allow
backslash-newline in // comments.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The Base class generates trivial wrappers that just call the underlying
function. It is meant as a base class to construct useful wrapper generators.
The Logging class generates wrappers that can log the inputs and outputs to
a function.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Some basic test coverage for now:
* Nominal operation.
* Larger output buffer.
* Clone an operation and use it after the original operation stops.
Generate test data automatically. For the time being, only do that for
hashes that Python supports natively. Supporting all algorithms is future
work.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
This will let us use these features from other modules (yet to be created).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
If HOSTCC is set, use that to generate files, otherwise use CC. This
should make cross-compilation with generated files slightly easier.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
Move compilation to a separate helper function in c_build_helper.py to
allow more generic use.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
This prevents a return type error in a later function that uses the
dictionaries here properly typed.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
We had 4 identical copies of the check_repo_path function. Replace them by a
single copy in the build_tree module where it naturally belongs.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
We were using absolute imports under the assumption that the /scripts
directory is in the path. This worked in normal use because every one of our
Python scripts either were in the /scripts directory, or added the /scripts
directory to the module search path in order to reference mbedtls_dev.
However, this broke things like
```
python3 -m unittest scripts/mbedtls_dev/psa_storage.py
```
Fix this by using relative imports.
Relative imports are only supposed to be used inside a package (Python
doesn't complain, but Pylint does). So make /scripts/mbedtls_dev a proper
package by creating __init__.py.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Do this in 2.28 just like it's done in the development branch, so that
code and command line usage that works on one branch doesn't surprisingly
fail on 2.28.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
test_suite_directory can be changed by a command line option in the
development branch but not in 2.28. Align the simplified version here with a
change in the development version
("generate_*_tests.py --directory: fix handling of relative path").
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
We have Python code both for test code generation
(tests/scripts/generate_test_code.py) and now for test data generation.
Avoid the ambiguous expression "test generation".
This commit renames the Python module and adjusts all references to it. A
subsequent commit will adjust the documentation.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Copy of scripts/mbedtls_dev/build_tree.py from mbedtls-3.2.1,
backported to facilitate future backports of python scripts.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When writing the new .data file, first write the new content, then replace
the target. This way, there isn't a temporary state in which the file is
partially written. This temporary state can be misleading if the build is
interrupted. It's annoying if you're watching changes to the output and the
changes appear as emptying the file following by the new version appearing.
Now interrupted builds don't leave a file that appears to be up to date but
isn't, and when watching the output, there's a single transition to the new
version.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Previous changes used the docstring of the test_generation module,
which does not inform a user about the script.
Signed-off-by: Werner Lewis <werner.lewis@arm.com>
BaseTarget-derived targets are now added to TestGenerator.targets in
initialization. This reduces repeated code in generate_xxx_tests.py
scripts which use this framework.
Signed-off-by: Werner Lewis <werner.lewis@arm.com>
