From 88a633da17d53261e0131e2d77209f22403112d6 Mon Sep 17 00:00:00 2001 From: David Steele Date: Tue, 24 Apr 2018 09:12:25 -0400 Subject: [PATCH] Divide tests into three types (unit, integration, performance). Many options that were set per test can instead be inferred from the types, i.e. container, c, expect, and individual. Also finish renaming Perl unit tests with the -perl suffix. --- doc/xml/release.xml | 4 + test/define.yaml | 144 ++++++--------- test/lib/pgBackRestTest/Common/DefineTest.pm | 165 ++++++++++-------- test/lib/pgBackRestTest/Common/ListTest.pm | 2 +- ...UnitTest.pm => ArchiveInfoUnitPerlTest.pm} | 4 +- ...oUnitTest.pm => BackupInfoUnitPerlTest.pm} | 4 +- ...ackupUnitTest.pm => BackupUnitPerlTest.pm} | 4 +- ...entTest.pm => CommonHttpClientPerlTest.pm} | 2 +- ...redTest.pm => CommonIoBufferedPerlTest.pm} | 4 +- ...cessTest.pm => CommonIoProcessPerlTest.pm} | 4 +- .../{InfoUnitTest.pm => InfoUnitPerlTest.pm} | 4 +- ...ifestAllTest.pm => ManifestAllPerlTest.pm} | 4 +- ...est.pm => ProtocolCommonMinionPerlTest.pm} | 4 +- ...elperTest.pm => ProtocolHelperPerlTest.pm} | 2 +- ...{StanzaAllTest.pm => StanzaAllPerlTest.pm} | 4 +- ...pm => StorageFilterCipherBlockPerlTest.pm} | 2 +- ...ipTest.pm => StorageFilterGzipPerlTest.pm} | 4 +- ...ShaTest.pm => StorageFilterShaPerlTest.pm} | 4 +- ...geLocalTest.pm => StorageLocalPerlTest.pm} | 4 +- ...gePosixTest.pm => StoragePosixPerlTest.pm} | 2 +- ...S3AuthTest.pm => StorageS3AuthPerlTest.pm} | 2 +- ...S3CertTest.pm => StorageS3CertPerlTest.pm} | 2 +- ...{StorageS3Test.pm => StorageS3PerlTest.pm} | 2 +- ...estTest.pm => StorageS3RequestPerlTest.pm} | 2 +- 24 files changed, 187 insertions(+), 192 deletions(-) rename test/lib/pgBackRestTest/Module/Archive/{ArchiveInfoUnitTest.pm => ArchiveInfoUnitPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Backup/{BackupInfoUnitTest.pm => BackupInfoUnitPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Backup/{BackupUnitTest.pm => BackupUnitPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Common/{CommonHttpClientTest.pm => CommonHttpClientPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Common/{CommonIoBufferedTest.pm => CommonIoBufferedPerlTest.pm} (98%) rename test/lib/pgBackRestTest/Module/Common/{CommonIoProcessTest.pm => CommonIoProcessPerlTest.pm} (96%) rename test/lib/pgBackRestTest/Module/Info/{InfoUnitTest.pm => InfoUnitPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Manifest/{ManifestAllTest.pm => ManifestAllPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Protocol/{ProtocolCommonMinionTest.pm => ProtocolCommonMinionPerlTest.pm} (97%) rename test/lib/pgBackRestTest/Module/Protocol/{ProtocolHelperTest.pm => ProtocolHelperPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Stanza/{StanzaAllTest.pm => StanzaAllPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Storage/{StorageFilterCipherBlockTest.pm => StorageFilterCipherBlockPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Storage/{StorageFilterGzipTest.pm => StorageFilterGzipPerlTest.pm} (98%) rename test/lib/pgBackRestTest/Module/Storage/{StorageFilterShaTest.pm => StorageFilterShaPerlTest.pm} (97%) rename test/lib/pgBackRestTest/Module/Storage/{StorageLocalTest.pm => StorageLocalPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Storage/{StoragePosixTest.pm => StoragePosixPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Storage/{StorageS3AuthTest.pm => StorageS3AuthPerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Storage/{StorageS3CertTest.pm => StorageS3CertPerlTest.pm} (98%) rename test/lib/pgBackRestTest/Module/Storage/{StorageS3Test.pm => StorageS3PerlTest.pm} (99%) rename test/lib/pgBackRestTest/Module/Storage/{StorageS3RequestTest.pm => StorageS3RequestPerlTest.pm} (99%) diff --git a/doc/xml/release.xml b/doc/xml/release.xml index ad57ad35f..841d2a229 100644 --- a/doc/xml/release.xml +++ b/doc/xml/release.xml @@ -198,6 +198,10 @@

Rename Perl tests so they don't conflict with their C counterparts.

+ +

Divide tests into three types (unit, integration, performance). Many options that were set per test can instead be inferred from the types, i.e. container, c, expect, and individual.

+ +

Try tweaking time sync settings to prevent clock drift rather than restarting VBoxService on every test run.

 diff --git a/test/define.yaml b/test/define.yaml index b6d639312..951b6ff3a 100644 --- a/test/define.yaml +++ b/test/define.yaml @@ -9,33 +9,34 @@ # # Most options can be set for modules and tests (test option will override module option if both are set): # * db - determines if the test will be run against multiple db versions -# * container - determines if the test will be run in a container or will create containers itself # * coverage - determines coverage for the test # full - the module/test provides full coverage for the code module # partial - the module/test provides partial coverage for the code module # noCode - the code module should not contain any coverable code. If it does an error will be thrown. -# * expect - should expect log tests be run? -# * c - is this a C test (instead of Perl)? -# * cDef - defines for C testing +# * define - defines for C testing # * debugUnitSuppress - don't define DEBUG_UNIT for unit tests -- this is used to test unit test debugging macros -# * individual - determines if each run in a test will be run in a new container # * perlReq - is Perl required for this C test? # # Some options are unique to tests: # * total - total runs in the test # * vm - VMs that the test will be run on # ********************************************************************************************************************************** -module: + +# ********************************************************************************************************************************** +# Unit tests +# +# Unit tests are assumed to be C tests unless they end in "-perl". +# ********************************************************************************************************************************** +unit: + # ******************************************************************************************************************************** - name: common - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - name: time total: 2 - c: true - cDef: -DNO_ERROR -DNO_LOG + define: -DNO_ERROR -DNO_LOG coverage: common/time: full @@ -43,8 +44,7 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: error total: 6 - c: true - cDef: -DNO_ERROR -DNO_LOG + define: -DNO_ERROR -DNO_LOG coverage: common/error: full @@ -53,8 +53,7 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: assert-on total: 2 - c: true - cDef: -DNO_LOG + define: -DNO_LOG coverage: common/assert: noCode @@ -62,8 +61,7 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: assert-off total: 2 - c: true - cDef: -DNDEBUG -DNO_LOG + define: -DNDEBUG -DNO_LOG debugUnitSuppress: true coverage: @@ -72,8 +70,7 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: fork total: 1 - c: true - cDef: -DNO_LOG + define: -DNO_LOG coverage: common/fork: full @@ -81,8 +78,7 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: mem-context total: 7 - c: true - cDef: -DNO_MEM_CONTEXT -DNO_LOG + define: -DNO_MEM_CONTEXT -DNO_LOG coverage: common/memContext: full @@ -90,8 +86,7 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: log total: 4 - c: true - cDef: -DNO_LOG + define: -DNO_LOG coverage: common/log: full @@ -99,7 +94,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: debug-on total: 2 - c: true coverage: common/debug: noCode @@ -107,8 +101,7 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: debug-off total: 2 - c: true - cDef: -DNDEBUG -DNO_LOG + define: -DNDEBUG -DNO_LOG debugUnitSuppress: true coverage: @@ -117,7 +110,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: lock total: 2 - c: true coverage: common/lock: full @@ -125,7 +117,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: io-handle total: 1 - c: true coverage: common/io/handle: full @@ -133,7 +124,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: exit total: 3 - c: true coverage: common/exit: full @@ -141,7 +131,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: wait total: 1 - c: true coverage: common/wait: full @@ -149,7 +138,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: type-list total: 3 - c: true coverage: common/type/list: full @@ -157,7 +145,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: type-string total: 9 - c: true coverage: common/type/string: full @@ -165,7 +152,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: type-string-list total: 7 - c: true coverage: common/type/stringList: full @@ -173,7 +159,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: type-buffer total: 4 - c: true coverage: common/type/buffer: full @@ -181,7 +166,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: type-variant total: 7 - c: true coverage: common/type/variant: full @@ -189,7 +173,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: type-variant-list total: 3 - c: true coverage: common/type/variantList: full @@ -197,7 +180,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: type-key-value total: 2 - c: true coverage: common/type/keyValue: full @@ -205,7 +187,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: encode total: 1 - c: true coverage: common/encode: full @@ -216,7 +197,7 @@ module: total: 1 # ---------------------------------------------------------------------------------------------------------------------------- - - name: http-client + - name: http-client-perl total: 2 coverage: @@ -225,7 +206,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: reg-exp total: 2 - c: true coverage: common/regExp: full @@ -233,7 +213,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: ini total: 3 - c: true coverage: common/ini: full @@ -253,14 +232,14 @@ module: Common/Io/Handle: full # ---------------------------------------------------------------------------------------------------------------------------- - - name: io-buffered + - name: io-buffered-perl total: 3 coverage: Common/Io/Buffered: partial # ---------------------------------------------------------------------------------------------------------------------------- - - name: io-process + - name: io-process-perl total: 3 coverage: @@ -275,13 +254,11 @@ module: # ******************************************************************************************************************************** - name: cipher - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - name: random total: 1 - c: true coverage: cipher/random: full @@ -289,33 +266,28 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: block total: 2 - c: true coverage: cipher/block: full # ******************************************************************************************************************************** - name: postgres - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - name: page-checksum total: 3 - c: true coverage: postgres/pageChecksum: full # ******************************************************************************************************************************** - name: perl - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - name: config total: 1 - c: true coverage: perl/config: full @@ -323,7 +295,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: exec total: 2 - c: true perlReq: true coverage: @@ -331,26 +302,22 @@ module: # ******************************************************************************************************************************** - name: help - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - name: help total: 4 - c: true coverage: command/help/help: full # ******************************************************************************************************************************** - name: config - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - name: define total: 2 - c: true coverage: config/define: full @@ -359,7 +326,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: config total: 3 - c: true coverage: config/config: full @@ -368,7 +334,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: parse total: 3 - c: true coverage: config/parse: full @@ -377,39 +342,37 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: load total: 1 - c: true coverage: config/load: full # ******************************************************************************************************************************** - name: storage - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - - name: filter-cipher-block + - name: filter-cipher-block-perl total: 2 coverage: Storage/Filter/CipherBlock: full # ---------------------------------------------------------------------------------------------------------------------------- - - name: filter-gzip + - name: filter-gzip-perl total: 3 coverage: Storage/Filter/Gzip: full # ---------------------------------------------------------------------------------------------------------------------------- - - name: filter-sha + - name: filter-sha-perl total: 2 coverage: Storage/Filter/Sha: full # ---------------------------------------------------------------------------------------------------------------------------- - - name: posix + - name: posix-perl total: 9 coverage: @@ -418,25 +381,25 @@ module: Storage/Posix/FileWrite: partial # ---------------------------------------------------------------------------------------------------------------------------- - - name: s3-auth + - name: s3-auth-perl total: 5 coverage: Storage/S3/Auth: full # ---------------------------------------------------------------------------------------------------------------------------- - - name: s3-cert + - name: s3-cert-perl total: 1 # ---------------------------------------------------------------------------------------------------------------------------- - - name: s3-request + - name: s3-request-perl total: 2 coverage: Storage/S3/Request: partial # ---------------------------------------------------------------------------------------------------------------------------- - - name: s3 + - name: s3-perl total: 7 coverage: @@ -451,7 +414,7 @@ module: - d8 # ---------------------------------------------------------------------------------------------------------------------------- - - name: local + - name: local-perl total: 10 coverage: @@ -467,7 +430,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: file total: 3 - c: true coverage: storage/driver/posix/driverFile: full @@ -479,7 +441,6 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: storage total: 13 - c: true coverage: storage/driver/posix/driver: full @@ -488,25 +449,23 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: helper total: 3 - c: true coverage: storage/helper: full # ******************************************************************************************************************************** - name: protocol - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - - name: common-minion + - name: common-minion-perl total: 1 coverage: Protocol/Base/Minion: partial # ---------------------------------------------------------------------------------------------------------------------------- - - name: helper + - name: helper-perl total: 2 coverage: @@ -514,11 +473,10 @@ module: # ******************************************************************************************************************************** - name: info - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - - name: unit + - name: unit-perl total: 2 coverage: @@ -526,20 +484,17 @@ module: # ******************************************************************************************************************************** - name: command - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - name: command total: 1 - c: true coverage: command/command: full # ******************************************************************************************************************************** - name: archive - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- @@ -562,14 +517,13 @@ module: # ---------------------------------------------------------------------------------------------------------------------------- - name: push total: 2 - c: true perlReq: true coverage: command/archive/push/push: full # ---------------------------------------------------------------------------------------------------------------------------- - - name: info-unit + - name: info-unit-perl total: 4 coverage: @@ -585,18 +539,17 @@ module: # ******************************************************************************************************************************** - name: backup - container: true coverage: Backup/Common: full test: # ---------------------------------------------------------------------------------------------------------------------------- - - name: unit + - name: unit-perl total: 3 # ---------------------------------------------------------------------------------------------------------------------------- - - name: info-unit + - name: info-unit-perl total: 3 coverage: @@ -604,11 +557,10 @@ module: # ******************************************************************************************************************************** - name: manifest - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - - name: all + - name: all-perl total: 9 coverage: @@ -616,20 +568,25 @@ module: # ******************************************************************************************************************************** - name: stanza - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- - - name: all + - name: all-perl total: 9 coverage: Stanza: full +# ********************************************************************************************************************************** +# Integration tests +# +# Integration tests are not run in a container. They are expected to create their own containers since most integration runs will +# create more than one. For this reason each run is executed individually. +# ********************************************************************************************************************************** +integration: + # ******************************************************************************************************************************** - name: mock - individual: true - expect: true test: # ---------------------------------------------------------------------------------------------------------------------------- @@ -654,8 +611,6 @@ module: # ******************************************************************************************************************************** - name: real - individual: true - expect: true db: true test: @@ -663,9 +618,16 @@ module: - name: all total: 6 +# ********************************************************************************************************************************** +# Performance tests +# +# Performance tests run in a single container but are more like integration tests than unit tests since they call the pgbackrest +# executable directly. +# ********************************************************************************************************************************** +performance: + # ******************************************************************************************************************************** - name: performance - container: true test: # ---------------------------------------------------------------------------------------------------------------------------- diff --git a/test/lib/pgBackRestTest/Common/DefineTest.pm b/test/lib/pgBackRestTest/Common/DefineTest.pm index 166dfca6e..d377fdc1d 100644 --- a/test/lib/pgBackRestTest/Common/DefineTest.pm +++ b/test/lib/pgBackRestTest/Common/DefineTest.pm @@ -23,6 +23,13 @@ use pgBackRestTest::Common::VmTest; # # Documentation for these constants is in test/define.yaml. ################################################################################################################################ +use constant TESTDEF_INTEGRATION => 'integration'; + push @EXPORT, qw(TESTDEF_INTEGRATION); +use constant TESTDEF_PERFORMANCE => 'performance'; + push @EXPORT, qw(TESTDEF_PERFORMANCE); +use constant TESTDEF_UNIT => 'unit'; + push @EXPORT, qw(TESTDEF_UNIT); + use constant TESTDEF_MODULE => 'module'; push @EXPORT, qw(TESTDEF_MODULE); use constant TESTDEF_NAME => 'name'; @@ -40,8 +47,8 @@ use constant TESTDEF_EXPECT => 'expect'; push @EXPORT, qw(TESTDEF_EXPECT); use constant TESTDEF_C => 'c'; push @EXPORT, qw(TESTDEF_C); -use constant TESTDEF_CDEF => 'cDef'; - push @EXPORT, qw(TESTDEF_CDEF); +use constant TESTDEF_DEFINE => 'define'; + push @EXPORT, qw(TESTDEF_DEFINE); use constant TESTDEF_DEBUG_UNIT_SUPPRESS => 'debugUnitSuppress'; push @EXPORT, qw(TESTDEF_DEBUG_UNIT_SUPPRESS); use constant TESTDEF_INDIVIDUAL => 'individual'; @@ -79,78 +86,100 @@ sub testDefLoad my $hTestDef = Load($strDefineYaml); - # Iterate each module - foreach my $hModule (@{$hTestDef->{&TESTDEF_MODULE}}) + # Iterate each test type + foreach my $strModuleType (TESTDEF_UNIT, TESTDEF_INTEGRATION, TESTDEF_PERFORMANCE) { - # Push the module onto the ordered list - my $strModule = $hModule->{&TESTDEF_NAME}; - push(@stryModule, $strModule); + my $hModuleType = $hTestDef->{$strModuleType}; - # Iterate each test - my @stryModuleTest; + my $bExpect = false; # By default don't run expect tests + my $bContainer = true; # By default run tests in a single container + my $bIndividual = false; # By default runs are all executed in the same contanier - foreach my $hModuleTest (@{$hModule->{&TESTDEF_TEST}}) + if ($strModuleType eq TESTDEF_INTEGRATION) { - # Push the test on the order list - my $strTest = $hModuleTest->{&TESTDEF_NAME}; - push(@stryModuleTest, $strTest); - - # Resolve variables that can be set in the module or the test - foreach my $strVar ( - TESTDEF_C, TESTDEF_CDEF, TESTDEF_CONTAINER, TESTDEF_DEBUG_UNIT_SUPPRESS, TESTDEF_DB, TESTDEF_EXPECT, - TESTDEF_INDIVIDUAL, TESTDEF_PERL_REQ, TESTDEF_VM) - { - $hTestDefHash->{$strModule}{$strTest}{$strVar} = coalesce( - $hModuleTest->{$strVar}, $hModule->{$strVar}, $strVar eq TESTDEF_VM ? undef : false); - - # Make false = 0 for debugging - if ($strVar ne TESTDEF_VM && $hTestDefHash->{$strModule}{$strTest}{$strVar} eq '') - { - $hTestDefHash->{$strModule}{$strTest}{$strVar} = false; - } - } - - # Set test count - $hTestDefHash->{$strModule}{$strTest}{&TESTDEF_TOTAL} = $hModuleTest->{&TESTDEF_TOTAL}; - - # If this is a C test then add the test module to coverage - if ($hModuleTest->{&TESTDEF_C}) - { - my $strTestFile = "module/${strModule}/${strTest}Test"; - - $hModuleTest->{&TESTDEF_COVERAGE}{$strTestFile} = TESTDEF_COVERAGE_FULL; - } - - # Concatenate coverage for modules and tests - foreach my $hCoverage ($hModule->{&TESTDEF_COVERAGE}, $hModuleTest->{&TESTDEF_COVERAGE}) - { - foreach my $strCodeModule (sort(keys(%{$hCoverage}))) - { - if (defined($hTestDefHash->{$strModule}{$strTest}{&TESTDEF_COVERAGE}{$strCodeModule})) - { - confess &log(ASSERT, - "${strCodeModule} is defined for coverage in both module ${strModule} and test ${strTest}"); - } - - $hTestDefHash->{$strModule}{$strTest}{&TESTDEF_COVERAGE}{$strCodeModule} = $hCoverage->{$strCodeModule}; - - # Build coverage type hash and make sure coverage type does not change - if (!defined($hCoverageType->{$strCodeModule})) - { - $hCoverageType->{$strCodeModule} = $hCoverage->{$strCodeModule}; - } - elsif ($hCoverageType->{$strCodeModule} ne $hCoverage->{$strCodeModule}) - { - confess &log(ASSERT, "cannot mix coverage types for ${strCodeModule}"); - } - - # Add to coverage list - push(@{$hCoverageList->{$strCodeModule}}, {strModule=> $strModule, strTest => $strTest}); - } - } + $bExpect = true; # Integration tests run expect tests + $bContainer = false; # Integration tests can run in multiple containers + $bIndividual = true; # Integration tests can change containers on each run } - $hModuleTest->{$strModule} = \@stryModuleTest; + # Iterate each module + foreach my $hModule (@{$hModuleType}) + { + # Push the module onto the ordered list + my $strModule = $hModule->{&TESTDEF_NAME}; + push(@stryModule, $strModule); + + # Iterate each test + my @stryModuleTest; + + foreach my $hModuleTest (@{$hModule->{&TESTDEF_TEST}}) + { + # Push the test on the order list + my $strTest = $hModuleTest->{&TESTDEF_NAME}; + push(@stryModuleTest, $strTest); + + # Resolve variables that can be set in the module or the test + foreach my $strVar (TESTDEF_DEFINE, TESTDEF_DEBUG_UNIT_SUPPRESS, TESTDEF_DB, TESTDEF_PERL_REQ, TESTDEF_VM) + { + $hTestDefHash->{$strModule}{$strTest}{$strVar} = coalesce( + $hModuleTest->{$strVar}, $hModule->{$strVar}, $strVar eq TESTDEF_VM ? undef : false); + + # Make false = 0 for debugging + if ($strVar ne TESTDEF_VM && $hTestDefHash->{$strModule}{$strTest}{$strVar} eq '') + { + $hTestDefHash->{$strModule}{$strTest}{$strVar} = false; + } + } + + # Set module type variables + $hTestDefHash->{$strModule}{$strTest}{&TESTDEF_C} = + $strModuleType eq TESTDEF_UNIT && $strTest !~ /\-perl$/ ? true : false; + $hTestDefHash->{$strModule}{$strTest}{&TESTDEF_EXPECT} = $bExpect; + $hTestDefHash->{$strModule}{$strTest}{&TESTDEF_CONTAINER} = $bContainer; + $hTestDefHash->{$strModule}{$strTest}{&TESTDEF_INDIVIDUAL} = $bIndividual; + + # Set test count + $hTestDefHash->{$strModule}{$strTest}{&TESTDEF_TOTAL} = $hModuleTest->{&TESTDEF_TOTAL}; + + # If this is a C test then add the test module to coverage + if ($hModuleTest->{&TESTDEF_C}) + { + my $strTestFile = "module/${strModule}/${strTest}Test"; + + $hModuleTest->{&TESTDEF_COVERAGE}{$strTestFile} = TESTDEF_COVERAGE_FULL; + } + + # Concatenate coverage for modules and tests + foreach my $hCoverage ($hModule->{&TESTDEF_COVERAGE}, $hModuleTest->{&TESTDEF_COVERAGE}) + { + foreach my $strCodeModule (sort(keys(%{$hCoverage}))) + { + if (defined($hTestDefHash->{$strModule}{$strTest}{&TESTDEF_COVERAGE}{$strCodeModule})) + { + confess &log(ASSERT, + "${strCodeModule} is defined for coverage in both module ${strModule} and test ${strTest}"); + } + + $hTestDefHash->{$strModule}{$strTest}{&TESTDEF_COVERAGE}{$strCodeModule} = $hCoverage->{$strCodeModule}; + + # Build coverage type hash and make sure coverage type does not change + if (!defined($hCoverageType->{$strCodeModule})) + { + $hCoverageType->{$strCodeModule} = $hCoverage->{$strCodeModule}; + } + elsif ($hCoverageType->{$strCodeModule} ne $hCoverage->{$strCodeModule}) + { + confess &log(ASSERT, "cannot mix coverage types for ${strCodeModule}"); + } + + # Add to coverage list + push(@{$hCoverageList->{$strCodeModule}}, {strModule=> $strModule, strTest => $strTest}); + } + } + } + + $hModuleTest->{$strModule} = \@stryModuleTest; + } } } diff --git a/test/lib/pgBackRestTest/Common/ListTest.pm b/test/lib/pgBackRestTest/Common/ListTest.pm index a5d52f25e..e340e54cd 100644 --- a/test/lib/pgBackRestTest/Common/ListTest.pm +++ b/test/lib/pgBackRestTest/Common/ListTest.pm @@ -148,7 +148,7 @@ sub testListGet { &TEST_VM => $strTestOS, &TEST_C => coalesce($hTest->{&TESTDEF_C}, $hModule->{&TESTDEF_C}, false), - &TEST_CDEF => $hTest->{&TESTDEF_CDEF}, + &TEST_CDEF => $hTest->{&TESTDEF_DEFINE}, &TEST_DEBUG_UNIT_SUPPRESS => $hTest->{&TEST_DEBUG_UNIT_SUPPRESS}, &TEST_CONTAINER => defined($hTest->{&TESTDEF_CONTAINER}) ? $hTest->{&TESTDEF_CONTAINER} : $hModule->{&TESTDEF_CONTAINER}, diff --git a/test/lib/pgBackRestTest/Module/Archive/ArchiveInfoUnitTest.pm b/test/lib/pgBackRestTest/Module/Archive/ArchiveInfoUnitPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Archive/ArchiveInfoUnitTest.pm rename to test/lib/pgBackRestTest/Module/Archive/ArchiveInfoUnitPerlTest.pm index 86f74924b..d99ba55de 100644 --- a/test/lib/pgBackRestTest/Module/Archive/ArchiveInfoUnitTest.pm +++ b/test/lib/pgBackRestTest/Module/Archive/ArchiveInfoUnitPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# BackupInfoUnitTest.pm - Unit tests for BackupInfo +# Unit tests for ArchiveInfo #################################################################################################################################### -package pgBackRestTest::Module::Archive::ArchiveInfoUnitTest; +package pgBackRestTest::Module::Archive::ArchiveInfoUnitPerlTest; use parent 'pgBackRestTest::Env::HostEnvTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Backup/BackupInfoUnitTest.pm b/test/lib/pgBackRestTest/Module/Backup/BackupInfoUnitPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Backup/BackupInfoUnitTest.pm rename to test/lib/pgBackRestTest/Module/Backup/BackupInfoUnitPerlTest.pm index 4dfb8d743..decc27402 100644 --- a/test/lib/pgBackRestTest/Module/Backup/BackupInfoUnitTest.pm +++ b/test/lib/pgBackRestTest/Module/Backup/BackupInfoUnitPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# BackupInfoUnitTest.pm - Unit tests for BackupInfo +# Unit tests for BackupInfo #################################################################################################################################### -package pgBackRestTest::Module::Backup::BackupInfoUnitTest; +package pgBackRestTest::Module::Backup::BackupInfoUnitPerlTest; use parent 'pgBackRestTest::Env::HostEnvTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Backup/BackupUnitTest.pm b/test/lib/pgBackRestTest/Module/Backup/BackupUnitPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Backup/BackupUnitTest.pm rename to test/lib/pgBackRestTest/Module/Backup/BackupUnitPerlTest.pm index c5eba1b35..445d37091 100644 --- a/test/lib/pgBackRestTest/Module/Backup/BackupUnitTest.pm +++ b/test/lib/pgBackRestTest/Module/Backup/BackupUnitPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# BackupUnitTest.pm - Tests for Backup module +# Tests for Backup module #################################################################################################################################### -package pgBackRestTest::Module::Backup::BackupUnitTest; +package pgBackRestTest::Module::Backup::BackupUnitPerlTest; use parent 'pgBackRestTest::Env::HostEnvTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Common/CommonHttpClientTest.pm b/test/lib/pgBackRestTest/Module/Common/CommonHttpClientPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Common/CommonHttpClientTest.pm rename to test/lib/pgBackRestTest/Module/Common/CommonHttpClientPerlTest.pm index 4ae7b2eb3..631de8c23 100644 --- a/test/lib/pgBackRestTest/Module/Common/CommonHttpClientTest.pm +++ b/test/lib/pgBackRestTest/Module/Common/CommonHttpClientPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### # S3 Request Tests #################################################################################################################################### -package pgBackRestTest::Module::Common::CommonHttpClientTest; +package pgBackRestTest::Module::Common::CommonHttpClientPerlTest; use parent 'pgBackRestTest::Common::RunTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Common/CommonIoBufferedTest.pm b/test/lib/pgBackRestTest/Module/Common/CommonIoBufferedPerlTest.pm similarity index 98% rename from test/lib/pgBackRestTest/Module/Common/CommonIoBufferedTest.pm rename to test/lib/pgBackRestTest/Module/Common/CommonIoBufferedPerlTest.pm index b2d64db82..91b1eeae5 100644 --- a/test/lib/pgBackRestTest/Module/Common/CommonIoBufferedTest.pm +++ b/test/lib/pgBackRestTest/Module/Common/CommonIoBufferedPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# CommonIoBufferedTest.pm - tests for Common::Io::Buffered module +# Tests for Common::Io::Buffered module #################################################################################################################################### -package pgBackRestTest::Module::Common::CommonIoBufferedTest; +package pgBackRestTest::Module::Common::CommonIoBufferedPerlTest; use parent 'pgBackRestTest::Common::RunTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Common/CommonIoProcessTest.pm b/test/lib/pgBackRestTest/Module/Common/CommonIoProcessPerlTest.pm similarity index 96% rename from test/lib/pgBackRestTest/Module/Common/CommonIoProcessTest.pm rename to test/lib/pgBackRestTest/Module/Common/CommonIoProcessPerlTest.pm index ca47403a4..25cd8d7d5 100644 --- a/test/lib/pgBackRestTest/Module/Common/CommonIoProcessTest.pm +++ b/test/lib/pgBackRestTest/Module/Common/CommonIoProcessPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# CommonIoProcessTest.pm - tests for Common::Io::Process module +# Tests for Common::Io::Process module #################################################################################################################################### -package pgBackRestTest::Module::Common::CommonIoProcessTest; +package pgBackRestTest::Module::Common::CommonIoProcessPerlTest; use parent 'pgBackRestTest::Common::RunTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Info/InfoUnitTest.pm b/test/lib/pgBackRestTest/Module/Info/InfoUnitPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Info/InfoUnitTest.pm rename to test/lib/pgBackRestTest/Module/Info/InfoUnitPerlTest.pm index 19183c260..8f375075c 100644 --- a/test/lib/pgBackRestTest/Module/Info/InfoUnitTest.pm +++ b/test/lib/pgBackRestTest/Module/Info/InfoUnitPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# InfoUnitTest.pm - Unit tests for Info module +# Unit tests for Info module #################################################################################################################################### -package pgBackRestTest::Module::Info::InfoUnitTest; +package pgBackRestTest::Module::Info::InfoUnitPerlTest; use parent 'pgBackRestTest::Env::HostEnvTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Manifest/ManifestAllTest.pm b/test/lib/pgBackRestTest/Module/Manifest/ManifestAllPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Manifest/ManifestAllTest.pm rename to test/lib/pgBackRestTest/Module/Manifest/ManifestAllPerlTest.pm index 53ccfaee9..e8f0c6757 100644 --- a/test/lib/pgBackRestTest/Module/Manifest/ManifestAllTest.pm +++ b/test/lib/pgBackRestTest/Module/Manifest/ManifestAllPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# ManifestAllTest.pm - Unit tests for Manifest module +# Unit tests for Manifest module #################################################################################################################################### -package pgBackRestTest::Module::Manifest::ManifestAllTest; +package pgBackRestTest::Module::Manifest::ManifestAllPerlTest; use parent 'pgBackRestTest::Env::HostEnvTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Protocol/ProtocolCommonMinionTest.pm b/test/lib/pgBackRestTest/Module/Protocol/ProtocolCommonMinionPerlTest.pm similarity index 97% rename from test/lib/pgBackRestTest/Module/Protocol/ProtocolCommonMinionTest.pm rename to test/lib/pgBackRestTest/Module/Protocol/ProtocolCommonMinionPerlTest.pm index dd6ed2da0..21dde5948 100644 --- a/test/lib/pgBackRestTest/Module/Protocol/ProtocolCommonMinionTest.pm +++ b/test/lib/pgBackRestTest/Module/Protocol/ProtocolCommonMinionPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# ProtocolCommonMinionTest.pm - tests for Protocol::Common::Minion module +# Tests for Protocol::Common::Minion module #################################################################################################################################### -package pgBackRestTest::Module::Protocol::ProtocolCommonMinionTest; +package pgBackRestTest::Module::Protocol::ProtocolCommonMinionPerlTest; use parent 'pgBackRestTest::Common::RunTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Protocol/ProtocolHelperTest.pm b/test/lib/pgBackRestTest/Module/Protocol/ProtocolHelperPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Protocol/ProtocolHelperTest.pm rename to test/lib/pgBackRestTest/Module/Protocol/ProtocolHelperPerlTest.pm index 053018a45..90390a242 100644 --- a/test/lib/pgBackRestTest/Module/Protocol/ProtocolHelperTest.pm +++ b/test/lib/pgBackRestTest/Module/Protocol/ProtocolHelperPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### # Protocol Helper Tests #################################################################################################################################### -package pgBackRestTest::Module::Protocol::ProtocolHelperTest; +package pgBackRestTest::Module::Protocol::ProtocolHelperPerlTest; use parent 'pgBackRestTest::Env::ConfigEnvTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Stanza/StanzaAllTest.pm b/test/lib/pgBackRestTest/Module/Stanza/StanzaAllPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Stanza/StanzaAllTest.pm rename to test/lib/pgBackRestTest/Module/Stanza/StanzaAllPerlTest.pm index 981599192..c81105a3d 100644 --- a/test/lib/pgBackRestTest/Module/Stanza/StanzaAllTest.pm +++ b/test/lib/pgBackRestTest/Module/Stanza/StanzaAllPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# StanzaAllTest.pm - Unit tests for Stanza.pm +# Unit tests for Stanza module #################################################################################################################################### -package pgBackRestTest::Module::Stanza::StanzaAllTest; +package pgBackRestTest::Module::Stanza::StanzaAllPerlTest; use parent 'pgBackRestTest::Env::HostEnvTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Storage/StorageFilterCipherBlockTest.pm b/test/lib/pgBackRestTest/Module/Storage/StorageFilterCipherBlockPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Storage/StorageFilterCipherBlockTest.pm rename to test/lib/pgBackRestTest/Module/Storage/StorageFilterCipherBlockPerlTest.pm index 1486c5ff7..dad5fa9a6 100644 --- a/test/lib/pgBackRestTest/Module/Storage/StorageFilterCipherBlockTest.pm +++ b/test/lib/pgBackRestTest/Module/Storage/StorageFilterCipherBlockPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### # Tests for Block Cipher #################################################################################################################################### -package pgBackRestTest::Module::Storage::StorageFilterCipherBlockTest; +package pgBackRestTest::Module::Storage::StorageFilterCipherBlockPerlTest; use parent 'pgBackRestTest::Common::RunTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Storage/StorageFilterGzipTest.pm b/test/lib/pgBackRestTest/Module/Storage/StorageFilterGzipPerlTest.pm similarity index 98% rename from test/lib/pgBackRestTest/Module/Storage/StorageFilterGzipTest.pm rename to test/lib/pgBackRestTest/Module/Storage/StorageFilterGzipPerlTest.pm index 9e4aa895d..6671937c9 100644 --- a/test/lib/pgBackRestTest/Module/Storage/StorageFilterGzipTest.pm +++ b/test/lib/pgBackRestTest/Module/Storage/StorageFilterGzipPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# StorageFilterGzipTest.pm - Tests for Storage::Filter::Gzip module. +# Tests for Storage::Filter::Gzip module #################################################################################################################################### -package pgBackRestTest::Module::Storage::StorageFilterGzipTest; +package pgBackRestTest::Module::Storage::StorageFilterGzipPerlTest; use parent 'pgBackRestTest::Common::RunTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Storage/StorageFilterShaTest.pm b/test/lib/pgBackRestTest/Module/Storage/StorageFilterShaPerlTest.pm similarity index 97% rename from test/lib/pgBackRestTest/Module/Storage/StorageFilterShaTest.pm rename to test/lib/pgBackRestTest/Module/Storage/StorageFilterShaPerlTest.pm index ee385b9aa..e9929160f 100644 --- a/test/lib/pgBackRestTest/Module/Storage/StorageFilterShaTest.pm +++ b/test/lib/pgBackRestTest/Module/Storage/StorageFilterShaPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# StorageFilterShaTest.pm - Tests for StorageFilterSha module. +# Tests for StorageFilterSha module #################################################################################################################################### -package pgBackRestTest::Module::Storage::StorageFilterShaTest; +package pgBackRestTest::Module::Storage::StorageFilterShaPerlTest; use parent 'pgBackRestTest::Common::RunTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Storage/StorageLocalTest.pm b/test/lib/pgBackRestTest/Module/Storage/StorageLocalPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Storage/StorageLocalTest.pm rename to test/lib/pgBackRestTest/Module/Storage/StorageLocalPerlTest.pm index 39450695a..659d1999e 100644 --- a/test/lib/pgBackRestTest/Module/Storage/StorageLocalTest.pm +++ b/test/lib/pgBackRestTest/Module/Storage/StorageLocalPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### -# StorageLocalTest.pm - Tests for Storage::Local module +# Tests for Storage::Local module #################################################################################################################################### -package pgBackRestTest::Module::Storage::StorageLocalTest; +package pgBackRestTest::Module::Storage::StorageLocalPerlTest; use parent 'pgBackRestTest::Common::RunTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Storage/StoragePosixTest.pm b/test/lib/pgBackRestTest/Module/Storage/StoragePosixPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Storage/StoragePosixTest.pm rename to test/lib/pgBackRestTest/Module/Storage/StoragePosixPerlTest.pm index b2e3c9c06..6007ab4a4 100644 --- a/test/lib/pgBackRestTest/Module/Storage/StoragePosixTest.pm +++ b/test/lib/pgBackRestTest/Module/Storage/StoragePosixPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### # Posix Driver Tests #################################################################################################################################### -package pgBackRestTest::Module::Storage::StoragePosixTest; +package pgBackRestTest::Module::Storage::StoragePosixPerlTest; use parent 'pgBackRestTest::Common::RunTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Storage/StorageS3AuthTest.pm b/test/lib/pgBackRestTest/Module/Storage/StorageS3AuthPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Storage/StorageS3AuthTest.pm rename to test/lib/pgBackRestTest/Module/Storage/StorageS3AuthPerlTest.pm index eb3810f53..50bfcd36b 100644 --- a/test/lib/pgBackRestTest/Module/Storage/StorageS3AuthTest.pm +++ b/test/lib/pgBackRestTest/Module/Storage/StorageS3AuthPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### # S3 Authentication Tests #################################################################################################################################### -package pgBackRestTest::Module::Storage::StorageS3AuthTest; +package pgBackRestTest::Module::Storage::StorageS3AuthPerlTest; use parent 'pgBackRestTest::Common::RunTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Storage/StorageS3CertTest.pm b/test/lib/pgBackRestTest/Module/Storage/StorageS3CertPerlTest.pm similarity index 98% rename from test/lib/pgBackRestTest/Module/Storage/StorageS3CertTest.pm rename to test/lib/pgBackRestTest/Module/Storage/StorageS3CertPerlTest.pm index b0951cdcc..bcd437f3c 100644 --- a/test/lib/pgBackRestTest/Module/Storage/StorageS3CertTest.pm +++ b/test/lib/pgBackRestTest/Module/Storage/StorageS3CertPerlTest.pm @@ -3,7 +3,7 @@ # # Verify that SSL certificate validation works on live S3 servers. #################################################################################################################################### -package pgBackRestTest::Module::Storage::StorageS3CertTest; +package pgBackRestTest::Module::Storage::StorageS3CertPerlTest; use parent 'pgBackRestTest::Env::ConfigEnvTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Storage/StorageS3Test.pm b/test/lib/pgBackRestTest/Module/Storage/StorageS3PerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Storage/StorageS3Test.pm rename to test/lib/pgBackRestTest/Module/Storage/StorageS3PerlTest.pm index 6015cbc9d..e1af2f62b 100644 --- a/test/lib/pgBackRestTest/Module/Storage/StorageS3Test.pm +++ b/test/lib/pgBackRestTest/Module/Storage/StorageS3PerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### # S3 Storage Tests #################################################################################################################################### -package pgBackRestTest::Module::Storage::StorageS3Test; +package pgBackRestTest::Module::Storage::StorageS3PerlTest; use parent 'pgBackRestTest::Env::S3EnvTest'; #################################################################################################################################### diff --git a/test/lib/pgBackRestTest/Module/Storage/StorageS3RequestTest.pm b/test/lib/pgBackRestTest/Module/Storage/StorageS3RequestPerlTest.pm similarity index 99% rename from test/lib/pgBackRestTest/Module/Storage/StorageS3RequestTest.pm rename to test/lib/pgBackRestTest/Module/Storage/StorageS3RequestPerlTest.pm index 7c69778b8..ba083fb02 100644 --- a/test/lib/pgBackRestTest/Module/Storage/StorageS3RequestTest.pm +++ b/test/lib/pgBackRestTest/Module/Storage/StorageS3RequestPerlTest.pm @@ -1,7 +1,7 @@ #################################################################################################################################### # S3 Request Tests #################################################################################################################################### -package pgBackRestTest::Module::Storage::StorageS3RequestTest; +package pgBackRestTest::Module::Storage::StorageS3RequestPerlTest; use parent 'pgBackRestTest::Common::RunTest'; ####################################################################################################################################