With GnuTLS servers, 3DES-CBC cipher suites are enabled by default under our
GNUTLS_LEGACY (3.3.8), but disabled by default under more recent versions
including the one we use by default on the CI (3.4.6). Even modern
versions (I checked 3.7.2) support 3DES if explicitly enabled. So
unconditionally enable 3DES-CBC for GnuTLS.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Alert if all tests are filtered out or skipped: that probably indicates a
test script that set up an unintended configuration or an overly strict
filter. You can pass `--min 0` to bypass this check. You can pass `--min`
with a larger value to require that many test cases to run.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Skipping DTLS 1.2 with old versions was already done, but now properly
test support only once and use the results.
Skipping single-DES with new versions is new, but helps finding the right
incantation.
Note that historically, this script's policy was that it's the user's
job to find the right value of -e (EXCLUDE) for their version for
OpenSSL & config. Now it's a weird mix of that and the script doing some
detection and skipping.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This is officially a 1.2-only ciphersuite, but we also support it with
1.0 and 1.1. However we don't support it with SSLv3 (see definition in
ssl_ciphersuites.c: mininum minor version is 1, that is TLS 1.0).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The main fix here is that m->O and m->G should use `M_CIPHERS`. In order
to apply that though, we need to change the structure with a new for
loop and case statement. The new structure matches what's done when
actually running tests.
Note: this issue only exists in 2.28. In 3.x we now use the standard
name for display everywhere, but in 2.28 we use the name as seen by the
client for display.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
We always want to call 'filter' regardless of the values of FILTER
and EXCLUDE because it also performs standard-defined filtering like
removing RC4 ciphersuites with DTLS.
(AFAICS, not calling 'filter' when we thought it was not needed was
just a performance optimisation.)
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
TLS-RSA-WITH-NULL-SHA256 is officially a cipher suite only since TLS 1.2,
but Mbed TLS allows it in 1.0/1.1, and so did ancient versions of GnuTLS.
This script was only considering it for GnuTLS interoperability if the
running version of GnuTLS supported it with older protocol versions.
Change to always considering it, and in particular always including it with
--list-test-cases, but skipping it at runtime if GnuTLS doesn't support it.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
walk_compat_sh and walk_opt_sh are basically the same now, so:
* Merge them into one function.
* Use the --list-test-cases option for both of them.
* Rename this merged function as collect_from_script which seems
more appropriate as since it isn't iterating the script but
calling it.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The official spelling of the trade mark changed from all-lowercase "mbed"
to normal proper noun capitalization "Mbed" a few years ago. We've been
using the new spelling in new text but still have the old spelling in a
lot of text. This commit updates most occurrences of "mbed TLS":
```
sed -i -e 's/mbed TLS/Mbed TLS/g' $(git ls-files ':!ChangeLog' ':!tests/data_files/**' ':!tests/suites/*.data' ':!programs/x509/*' ':!configs/tfm*')
```
Justification for the omissions:
* `ChangeLog`: historical text.
* `test/data_files/**`, `tests/suites/*.data`, `programs/x509/*`: many
occurrences are significant names in certificates and such. Changing
the spelling would invalidate many signatures and tests.
* `configs/tfm*`: this is an imported file. We'll follow the upstream
updates.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
TLS-RSA-WITH-NULL-SHA256, like other SHA256-based cipher suites, was first
introduced in TLS 1.2. Mbed TLS accepts it in earlier protocol versions as
well. This is technically a bug, which older versions of GnuTLS also have.
GnuTLS 3.4.7 fixed this bug. Adapt compat.sh to automatically omit
TLS-RSA-WITH-NULL-SHA256 in invalid protocol versions if GnuTLS doesn't
support it. It's already not included in invalid protocol versions in
OpenSSL interoperability testing.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When calling `add_xxx_ciphersuites`, we have to set MODE properly.
This commit adjusts order to address this issue in list_test_case
which matches what we do in a normal execution.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
This commit includes:
- use subprocess.check_output to report error and capture return
value
- add comment as a reminder for option --list-test-case
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
uniform_title is used to print identical format of $TITLE between
--list-test-case and run_client. In such way, no matter how $TITLE
is developed, --list-test-case will in the same format of test case
description as stored in OUTCOME.CSV.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Test case description is printed by different block of code. This
causes code maintenance harder since we need to maintain two parts
of code with same functionality. print_test_title is used to
control test case description in compat.sh
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
The option --list-test-case lists all potential test cases without
executing them. The test case description is identical with $TITLE
during test case execution.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
If the environment variable MBEDTLS_TEST_OUTCOME_FILE is set,
the test outcome file records each test case in a single line
with the format of
PLATFORM;CONFIGURATION;compat;TEST CASE DESCRIPTION;RESULT;[CAUSE]
- CONFIGURATION comes from MBEDTLS_TEST_CONFIGURATION to record
configuration of each test case
- PLATFORM is either set by users or calculated from test
platform
- RESULT is one of PASS, FAIL or SKIP. If test case fails,
srv_out/cli_out follows as FAILURE CAUSE.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
This commit add support to detect if openssl used for testing
supports static ECDH key exchange. Skip the ciphersutes if
openssl doesn't support them.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
Since PSK cipher suites do not allow client certificate verification,
PSK test cases should be executed under VERIFY=NO. SUB_VERIFIES is
used to constrain verification option for PSK tests.
With aforementioned change, the latter check of
$VERIFY=YES && $TYPE!=PSK is redundant so it's removed.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
With the introduction of PSK_TESTS,
- Either `compat.sh -V NO` or `compat.sh -V YES` runs the PSK tests
- `compat.sh` or `compat.sh -V "NO YES"` runs PSK tests only once
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
There is no need to provide CA file in PSK. Thus VERIFY is
meaningless for PSK. This change omits the arguments passed to
the client and server for $VERIFY=YES.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Under Ubuntu-22.04, wait command prints out Terminated message.
Therefore server process is handled with identical ways like other
processes in compat.sh. In addition, PROCESS_ID is renamed as
SRV_PID to improve code readability.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Under Ubuntu-22.04, wait command prints out Terminated message
if the process has been killed by kill command. This messes up
the output in compat.sh
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
These variables were both uses to select the default version of OpenSSL
to use for tests:
- when running compat.sh or ssl-opt.sh directly, OPENSSL_CMD was used;
- when running all.sh, OPENSSL was used.
This caused surprising situations if you had one but not the other set
in your environment. For example I used to have OPENSSL_CMD set but not
OPENSSL, so ssl-opt.sh was failing in some all.sh components but passing
when I ran it manually in the same configuration and build, a rather
unpleasant experience.
The natural name would be OPENSSL, and that's what set in the Docker
images used by the CI. However back in the 1.3.x days, that name was
already used in library/Makefile, so it was preferable to pick a
different one, hence OPENSSL_CMD. However the build system has not been
using this name since at least Mbed TLS 2.0.0, so it's now free for use
again (as demonstrated by the fact that it's been set in the CI without
causing any trouble).
So, unify things and use OPENSSL everywhere. Just leave an error message
for the benefit of developers which might have OPENSSL_CMD, not OPENSSL,
set in their environment from the old days.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Restore compatibiltiy testing against OpenSSL for
(D)TLS versions smaller that 1.2.
. Fix the check for support in OpenSSL for these versions
. For test cases for (D)TLS version smaller than 1.2,
restore the configuration of OpenSSL client/server
with the given TLS version, as it was before #5660
that broke it.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
OpenSSL may be configured to support features such as cipher suites or
protocol versions that are disabled by default. Enable them all: we're
testing, we don't care about enabling insecure stuff. This is not needed
with the builds of OpenSSL that we're currently using on the Jenkins CI, but
it's needed with more recent versions such as typically found on developer
machines, and with future CI additions.
The syntax to do that was only introduced in OpenSSL 1.1.0; fortunately we
don't need to do anything special with earlier versions.
With OpenSSL 1.1.1f on Ubuntu 20.04, this is needed to enable TLS 1.0, TLS
1.1 and DTLS 1.0. This also allows SHA-1 in certificates, which is still
needed for a few test cases in ssl-opt.sh. Curiously, this is also needed
for the cipher suite TLS-DHE-PSK-WITH-ARIA-128-GCM-SHA256 (and no other,
including other DHE-PSK or ARIA cipher suites).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
