lib/mbedtls
1
0
Fork 0
mirror of https://github.com/Mbed-TLS/mbedtls.git synced 2025-11-03 20:33:16 +03:00
Code Activity

test-framework.md: document compat.sh

Browse Source 
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
This commit is contained in:
Yanray Wang
2023-02-20 16:18:56 +08:00
committed by Tomás González
parent 44ba969acb
commit 31d5046f32
1 changed files with 9 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/architecture/testing/test-framework.md
View File

@@ -30,17 +30,23 @@ Each test case has a description which succinctly describes for a human audience
#### SSL test case descriptions #### SSL test case descriptions
Each test case in `ssl-opt.sh` has a description which succinctly describes for a human audience what the test does. The test description is the first parameter to `run_tests`. Each test case in `ssl-opt.sh` has a description which succinctly describes for a human audience what the test does. The test description is the first parameter to `run_test`.
The same rules and guidelines apply as for [unit test descriptions](#unit-test-descriptions). In addition, the description must be written on the same line as `run_test`, in double quotes, for the sake of `check_test_cases.py`. The same rules and guidelines apply as for [unit test descriptions](#unit-test-descriptions). In addition, the description must be written on the same line as `run_test`, in double quotes, for the sake of `check_test_cases.py`.
### SSL cipher suite tests
Each test case in `compat.sh` has a description which succinctly describes for a human audience what the test does. The test description is `$TITLE` defined in `run_client`.
The same rules and guidelines apply as for [unit test descriptions](#unit-test-descriptions). In addition, failure cause in `compat.sh` is not classified as `ssl-opt.sh`, so the information of failed log files are followed as prompt.
## Running tests ## Running tests
### Outcome file ### Outcome file
#### Generating an outcome file #### Generating an outcome file
Unit tests and `ssl-opt.sh` record the outcome of each test case in a **test outcome file**. This feature is enabled if the environment variable `MBEDTLS_TEST_OUTCOME_FILE` is set. Set it to the path of the desired file. Unit tests, `ssl-opt.sh` and `compat.sh` record the outcome of each test case in a **test outcome file**. This feature is enabled if the environment variable `MBEDTLS_TEST_OUTCOME_FILE` is set. Set it to the path of the desired file.
If you run `all.sh --outcome-file test-outcome.csv`, this collects the outcome of all the test cases in `test-outcome.csv`. If you run `all.sh --outcome-file test-outcome.csv`, this collects the outcome of all the test cases in `test-outcome.csv`.
@@ -52,7 +58,7 @@ The outcome file has 6 fields:
* **Platform**: a description of the platform, e.g. `Linux-x86_64` or `Linux-x86_64-gcc7-msan`. * **Platform**: a description of the platform, e.g. `Linux-x86_64` or `Linux-x86_64-gcc7-msan`.
* **Configuration**: a unique description of the configuration (`config.h`). * **Configuration**: a unique description of the configuration (`config.h`).
* **Test suite**: `test_suite_xxx` or `ssl-opt`. * **Test suite**: `test_suite_xxx`, `ssl-opt` or `compat`.
* **Test case**: the description of the test case. * **Test case**: the description of the test case.
* **Result**: one of `PASS`, `SKIP` or `FAIL`. * **Result**: one of `PASS`, `SKIP` or `FAIL`.
* **Cause**: more information explaining the result. * **Cause**: more information explaining the result.