Przemek Stekiel
85c54ea361
Allow providing space sepatated tasks
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-11-17 11:50:23 +01:00
Przemek Stekiel
542d932352
Fix handling of default value for task argument
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-11-17 09:43:34 +01:00
Przemek Stekiel
6419ab5299
Reduce number of skipped suites (after making configs more similar)
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-11-16 14:56:49 +01:00
Przemek Stekiel
d3068af2a8
Optimize code (tasks list initialization, task verification)
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-11-14 16:15:19 +01:00
Przemek Stekiel
733c76e08a
Fix style issues pointed by pylint
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-11-14 08:33:21 +01:00
Przemek Stekiel
992de3c562
Make TASK parameter positional and allow more than one task
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-11-09 14:24:49 +01:00
Przemek Stekiel
be279c7bcc
Make a list from ignored tests in TASKS
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-11-09 12:17:08 +01:00
Przemek Stekiel
51f30ff6e6
Make separate components for ref and driver in TASKS
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-11-09 12:07:29 +01:00
Przemek Stekiel
6856f4c70d
Fix typos and comments
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-11-09 10:50:29 +01:00
Przemek Stekiel
4d13c833da
analyze_outcomes.py: remove components and ignore parameters
...
Use a dictionary to specify optional parameters for each task.
If the task is not specified then all tasks are executed.
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-10-26 16:12:01 +02:00
Przemek Stekiel
c86dedfdc1
Fix code style
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-10-24 09:16:04 +02:00
Przemek Stekiel
58bbc23ca3
Use coverage analyze as default task
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-10-24 08:10:10 +02:00
Przemek Stekiel
4e95590ae7
analyze_outcomes.py: Add test coverage regresion analyze for driver only builds
...
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com >
2022-10-21 13:43:13 +02:00
Gilles Peskine
686c292e8a
Move collect_available_test_cases to check_test_cases.py
...
No behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com >
2022-01-07 18:34:12 +01:00
Gilles Peskine
3d863f2631
Document the fields of TestCasesOutcomes
...
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com >
2020-06-26 18:31:26 +02:00
Gilles Peskine
8d3c70a279
Check test case coverage
...
Check that every available test case in the test suites and ssl-opt.sh
has been executed at least once.
For the time being, only report a warning, because our coverage is
incomplete. Once we've updated all.sh to have full coverage, this
warning should become an error.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com >
2020-06-26 18:29:34 +02:00
Gilles Peskine
15c2cbfed5
New script for test outcome analysis
...
This is a new script designed to analyze test outcomes collected
during a whole CI run.
This commit introduces the script, the code to read the outcome file,
and a very simple framework to report errors. It does not perform any
actual analysis yet.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com >
2020-06-26 18:29:34 +02:00
