Manuel Pégourié-Gonnard 
							
						 
					 
					
						
						
							
						
						4da369f741 
					 
					
						
						
							
							analyze_outcomes: minor code cleanup  
						
						... 
						
						
						
						Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com > 
						
						
					 
					
						2023-10-27 09:17:35 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						507e08f9af 
					 
					
						
						
							
							analyze_outcomes: update cipher/aead data  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-26 10:05:01 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						97454fde54 
					 
					
						
						
							
							all.sh: accelerate CMAC in test_psa_crypto_config_accel_cipher_aead()  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-25 12:27:12 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						3b1559060a 
					 
					
						
						
							
							test_suite_psa_crypto: replace builtin dependencies with PSA_WANT  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-25 12:03:36 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						5cd18f9150 
					 
					
						
						
							
							analyze_oucomes.py: ignore line-too-long error for skipped tests  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-25 12:03:36 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						93941440c1 
					 
					
						
						
							
							all.sh: keep PKCS5/12 enabled in the reference component  
						
						... 
						
						
						
						This commit also add proper "ignore" fields to the "analyze_outcomes.py"
script.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-25 12:03:36 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						7448cee8f0 
					 
					
						
						
							
							analyze_outcomes.py: skip tests that depends on builtin features  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-25 12:03:36 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						b6b301fa8d 
					 
					
						
						
							
							test: add component accelerating both ciphers and AEADs  
						
						... 
						
						
						
						This also adds a new task in analyze_outcomes.py for checking
the accelaration coverage against the reference element.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-25 12:03:36 +02:00 
						 
				 
			
				
					
						
							
							
								Yanray Wang 
							
						 
					 
					
						
						
							
						
						08e9423f14 
					 
					
						
						
							
							Merge remote-tracking branch 'origin/development' into adjust_tfm_configs  
						
						
						
						
					 
					
						2023-10-19 17:44:47 +08:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						2cff82069e 
					 
					
						
						
							
							analyze_outcomes: add new_section() method to the Results class  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-18 14:36:47 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						39d4b9d15b 
					 
					
						
						
							
							analyze_outcomes: fix format interpolation errors  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-18 14:30:03 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						735794c745 
					 
					
						
						
							
							analyze_outcomes: fix missing format for args/kwargs  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-18 08:05:15 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						781c23416e 
					 
					
						
						
							
							analyze_oucomes: do not return Results instance passed as parameter  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-17 13:07:10 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						8070dbec6b 
					 
					
						
						
							
							analyze_outcomes: keep print_line() method non-static  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-17 12:40:42 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						f6f64cfd81 
					 
					
						
						
							
							analyze_outcomes: code style improvement  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-17 12:28:26 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						8d178be66e 
					 
					
						
						
							
							analyze_outcomes: fix return value in case of test failure  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-17 12:23:55 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						40314fcc75 
					 
					
						
						
							
							analyze_outcomes: fix newlines  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-17 11:34:31 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						f075e47bc1 
					 
					
						
						
							
							analyze_outcomes: reset name of TestLog to Results  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-17 11:33:21 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						3f33989762 
					 
					
						
						
							
							analyze_outcomes: use a single TestLog instance and do not delay output  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-17 11:32:23 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						fb2750e98e 
					 
					
						
						
							
							analyze_outcomes: exit immediately in case of invalid task  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-17 10:57:53 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						5329ff06b9 
					 
					
						
						
							
							analyze_outcomes: print task list directly to stdout  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-17 09:44:36 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						b0c618e147 
					 
					
						
						
							
							analyze_outcomes: minor improvements  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-16 14:19:49 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						aaef0bc172 
					 
					
						
						
							
							analyze_outcomes: improve logging system  
						
						... 
						
						
						
						- the script now only terminates in case of hard faults
- each task is assigned a log
   - this log tracks messages, warning and errors
   - when task completes, errors and warnings are listed and
     messages are appended to the main log
- on exit the main log is printed and the proper return value
  is returned
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-12 09:16:40 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						dfd7ca6344 
					 
					
						
						
							
							analyze_outcomes: rename some variables for better readability  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-10-09 16:53:02 +02:00 
						 
				 
			
				
					
						
							
							
								Gilles Peskine 
							
						 
					 
					
						
						
							
						
						35b49c4d7d 
					 
					
						
						
							
							Ignore tests of built-in interfaces for driver-only testing parity  
						
						... 
						
						
						
						Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com > 
						
						
					 
					
						2023-10-04 12:28:41 +02:00 
						 
				 
			
				
					
						
							
							
								Manuel Pégourié-Gonnard 
							
						 
					 
					
						
						
							
						
						f07ce3b8ff 
					 
					
						
						
							
							Don't extend support for deprecated functions  
						
						... 
						
						
						
						Restore guards from the previous release, instead of the new, more
permissive guards.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com > 
						
						
					 
					
						2023-09-28 08:51:51 +02:00 
						 
				 
			
				
					
						
							
							
								Yanray Wang 
							
						 
					 
					
						
						
							
						
						89c88bb44b 
					 
					
						
						
							
							analyze_outcomes: fix incorrect use of Results.log()  
						
						... 
						
						
						
						Signed-off-by: Yanray Wang <yanray.wang@arm.com > 
						
						
					 
					
						2023-09-27 10:35:09 +08:00 
						 
				 
			
				
					
						
							
							
								Yanray Wang 
							
						 
					 
					
						
						
							
						
						5c0c858026 
					 
					
						
						
							
							analyze_outcomes: ignore asn1parse and asn1write in result analysis  
						
						... 
						
						
						
						By default, we disable ASN1_[PARSE/WRITE]_C in common_tfm_config.
In fact, this is what happens for accelerated p256m driver, which
means all asn1[parse/write] tests are skipped in driver_accel test.
However, those two macros are automatically enabled for built-in
ECDSA via PSA, which means all asn1[parse/write] tests are passed
in tfm_config test.
This commit simply ignores the whole asn1[parse/write] test suite
when analyzing between driver and reference.
Signed-off-by: Yanray Wang <yanray.wang@arm.com > 
						
						
					 
					
						2023-09-26 17:16:00 +08:00 
						 
				 
			
				
					
						
							
							
								Yanray Wang 
							
						 
					 
					
						
						
							
						
						0e319ae577 
					 
					
						
						
							
							analyze_outcomes: escape {} in string format for test description  
						
						... 
						
						
						
						{} are valid characters in test description, but they're not escaped
properly in python string format(). To resolve the bug of KeyError
when it tries to log test description which contains {}, we replace
{XXX} format with {{XXX}} in order to escape {} in python string
format() properly.
In addition, the calls to Results.log() are also handled to avoid
similar potential problems.
Signed-off-by: Yanray Wang <yanray.wang@arm.com > 
						
						
					 
					
						2023-09-26 17:16:00 +08:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						482a0b957f 
					 
					
						
						
							
							test: fix remaining disparities and remove debug leftovers  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-09-25 17:39:41 +02:00 
						 
				 
			
				
					
						
							
							
								Manuel Pégourié-Gonnard 
							
						 
					 
					
						
						
							
						
						5edb942708 
					 
					
						
						
							
							Merge pull request  #8041  from mpg/tfm-p256m  
						
						... 
						
						
						
						Test TF-M config with p256-m driver 
						
						
					 
					
						2023-09-20 16:09:56 +00:00 
						 
				 
			
				
					
						
							
							
								Manuel Pégourié-Gonnard 
							
						 
					 
					
						
						
							
						
						b95e92cd41 
					 
					
						
						
							
							Merge pull request  #8076  from valeriosetti/issue8005  
						
						... 
						
						
						
						Test with ECC and FFDH accelerated and no bignum 
						
						
					 
					
						2023-09-14 09:12:35 +00:00 
						 
				 
			
				
					
						
							
							
								Manuel Pégourié-Gonnard 
							
						 
					 
					
						
						
							
						
						e9d97976b2 
					 
					
						
						
							
							Update list of ignored tests  
						
						... 
						
						
						
						Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com > 
						
						
					 
					
						2023-09-12 09:50:17 +02:00 
						 
				 
			
				
					
						
							
							
								Tomás González 
							
						 
					 
					
						
						
							
						
						d43cab3f5c 
					 
					
						
						
							
							Correct analyze_outcomes identation  
						
						... 
						
						
						
						Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com > 
						
						
					 
					
						2023-08-24 09:12:40 +01:00 
						 
				 
			
				
					
						
							
							
								Tomás González 
							
						 
					 
					
						
						
							
						
						a0631446b5 
					 
					
						
						
							
							Correct analyze_outcomes.py identation  
						
						... 
						
						
						
						Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com > 
						
						
					 
					
						2023-08-22 12:18:04 +01:00 
						 
				 
			
				
					
						
							
							
								Tomás González 
							
						 
					 
					
						
						
							
						
						5022311c9d 
					 
					
						
						
							
							Tidy up allow list definition  
						
						... 
						
						
						
						* Don't break string literals in the allow list definition
 * Comment each test that belongs to the allow list is there.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com > 
						
						
					 
					
						2023-08-22 09:54:28 +01:00 
						 
				 
			
				
					
						
							
							
								Tomás González 
							
						 
					 
					
						
						
							
						
						7ebb18fbd6 
					 
					
						
						
							
							Make non-executed tests that are not in the allow list an error  
						
						... 
						
						
						
						* Turn the warnings produced when finding non-executed tests that
   are not in the allow list into errors.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com > 
						
						
					 
					
						2023-08-22 09:47:49 +01:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						307810babb 
					 
					
						
						
							
							analyze_outcomes: add case for "ECC+FFDH w/o BN"  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-08-16 08:01:17 +02:00 
						 
				 
			
				
					
						
							
							
								Tomás González 
							
						 
					 
					
						
						
							
						
						358c6c644a 
					 
					
						
						
							
							Add EdDSA and XTS to the allow list  
						
						... 
						
						
						
						As specified in
https://github.com/Mbed-TLS/mbedtls/issues/5390#issuecomment-1669585707 
EdDSA and XTS tests are legitimately never executed, so add them to
the allow list.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com > 
						
						
					 
					
						2023-08-14 15:46:25 +01:00 
						 
				 
			
				
					
						
							
							
								Tomás González 
							
						 
					 
					
						
						
							
						
						b401e113ff 
					 
					
						
						
							
							Add a flag for requiring full coverage in coverage tests  
						
						... 
						
						
						
						Introduce the --require-full-coverage in analyze_outcomes.py so that
when analyze_outcomes.py --require-full-coverage is called, those
tests that are not executed and are not in the allowed list issue an
error instead of a warning.
Note that it is useful to run analyze_outcomes.py on incomplete test
results, so this error mode needs to remain optional in the long
term.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com > 
						
						
					 
					
						2023-08-14 15:46:12 +01:00 
						 
				 
			
				
					
						
							
							
								Tomás González 
							
						 
					 
					
						
						
							
						
						07bdcc2b0d 
					 
					
						
						
							
							Add allow list for non-executed test cases  
						
						... 
						
						
						
						The allow list explicits which test cases are allowed to not be
executed when testing. This may be, for example, because a feature
is yet to be developed but the test for that feature is already in
our code base.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com > 
						
						
					 
					
						2023-08-14 15:46:12 +01:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						f01d648677 
					 
					
						
						
							
							analyze_outcome: add new check for parity for TFM configuration  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-08-11 08:33:27 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						e0be95e81d 
					 
					
						
						
							
							analyze_outcomes: skip tests that depend on BIGNUM_C  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-08-11 06:35:23 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						9b3dbcc2e3 
					 
					
						
						
							
							analyze_outcomes: skip tests that unavoidably depend on bignum  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-08-10 06:43:23 +02:00 
						 
				 
			
				
					
						
							
							
								Manuel Pégourié-Gonnard 
							
						 
					 
					
						
						
							
						
						abd00d0be8 
					 
					
						
						
							
							test: adding new components for testing and driver coverage analysis without BN  
						
						... 
						
						
						
						Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com >
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-08-10 06:43:23 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						5f54020d1a 
					 
					
						
						
							
							analyze_outcomes: do not skip test about wrong Montgomery private key  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-07-11 11:28:22 +02:00 
						 
				 
			
				
					
						
							
							
								Przemek Stekiel 
							
						 
					 
					
						
						
							
						
						565353ef71 
					 
					
						
						
							
							Cleanup the code  
						
						... 
						
						
						
						Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com > 
						
						
					 
					
						2023-07-05 11:07:07 +02:00 
						 
				 
			
				
					
						
							
							
								Przemek Stekiel 
							
						 
					 
					
						
						
							
						
						84f4ff1dd3 
					 
					
						
						
							
							Minor adaptations after ffdh was enabled for tls1.3  
						
						... 
						
						
						
						Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com > 
						
						
					 
					
						2023-07-05 09:12:08 +02:00 
						 
				 
			
				
					
						
							
							
								Przemek Stekiel 
							
						 
					 
					
						
						
							
						
						85b644262d 
					 
					
						
						
							
							Add ffdh accel vs reference check to analyze_outcomes.py  
						
						... 
						
						
						
						Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com > 
						
						
					 
					
						2023-07-04 12:35:54 +02:00 
						 
				 
			
				
					
						
							
							
								Valerio Setti 
							
						 
					 
					
						
						
							
						
						5bd2523178 
					 
					
						
						
							
							test: ignore compressed points' tests when checking coverage without ECP at all  
						
						... 
						
						
						
						Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no > 
						
						
					 
					
						2023-06-20 09:06:12 +02:00