2ba946b341
Add the "--fuzzdb FILENAME" to testrunner.tcl as an alternative to setting
...
the FUZZDB environment variable (as that can be awkward to do on Windows).
Further improvements to the testrunner.tcl documentation.
FossilOrigin-Name: 6fb84156a262ff89d1a2d1df6fbfac4c1a43fb55b9d15205508662e2c9b0894f
2025-04-29 14:23:21 +00:00
873fc5dff2
Update testrunner.tcl documentation. Fix typo in testrunner.tcl output.
...
FossilOrigin-Name: ba8800b4ba0a690c89d9fb96bb9a0d8789d04160ff9c47ea9afb2b403bec7f0b
2025-04-29 11:35:51 +00:00
c171d7624e
Improved --explain display of FUZZDB test plans in testrunner.tcl.
...
FossilOrigin-Name: 42bdd7262ec4f8163263e0535c2cc35b526754d77e1f3ccf5b73b0f44d470d9c
2025-04-29 10:41:16 +00:00
ece7709f18
Do not run FUZZDB on either Windows-Memdebug or Windows-Win32Heap.
...
FossilOrigin-Name: 10d8401d59136dda7051af0da398b06a18031560f2734f4fa347d339765d2089
2025-04-29 10:29:35 +00:00
8a68ab963e
Make testrunner.tcl more restrictive about which configurations run
...
fuzzcheck-asan and fuzzcheck-ubsan.
FossilOrigin-Name: 785599394dde0fb36ae2e07d611f9523a05a0817dbd8c416582b3b682dbfa4ff
2025-04-29 01:37:31 +00:00
246dc92354
Testrunner.tcl now runs fuzzcheck-asan and fuzzcheck-ubsan tests.
...
FossilOrigin-Name: 3d8f3a255843c063157265702b76d40c38c774d29d658eea2d77efac92ac8789
2025-04-28 19:15:46 +00:00
d4cc3a42e2
If the FUZZDB environment variable is set when running testrunner.tcl and
...
that env-var points to a fuzzcheck database, then suppliment the fuzzcheck
tests with that extra database.
FossilOrigin-Name: c08530dd6a71cb547d164e2313657c917f280e7988211bf199c8776e78a4e2a6
2025-04-28 17:56:23 +00:00
1365bcffbf
Enhance testrunner.tcl so that it does not build testfixture unnecessarily -
...
if the pattern does not match any TCL test.
FossilOrigin-Name: 896d767f1cba64bda7b91b0daebd60ebe4457a58d762da443bf61cf5b39ad2d9
2025-04-28 16:28:00 +00:00
7fec7c96c4
Fix testrunner.tcl so that it honors pattern matches in non-testfixture
...
tests, such as fuzzcheck and sessionfuzz.
FossilOrigin-Name: 81857474fe90f50b760c22d61a6ad477642dabe1052390cd1614dcec8adb1376
2025-04-28 16:17:17 +00:00
738443c523
Remove a stray blank line from the test/testrunner.tcl script.
...
FossilOrigin-Name: 7d6005b0d17d86d72757f3578872e82fd12f61ac9160a0277e397df09f6338a9
2025-04-28 15:00:28 +00:00
1588d3de37
More ==/!= => eq/ne fixes
...
FossilOrigin-Name: 2aa6c4fa6c9c47c7a8548ac3493bfb9ed4e4dec1b6dc4f3599a0373086b3e917
2025-03-27 14:37:07 +00:00
52e2a79304
Test script patches from Jan Nijtmans: always use -DUSE_TCL_STUBS in buildtclext.tcl and improvements in how TRG is populated in testrunner.tcl.
...
FossilOrigin-Name: aa192203aa1aa188866fe00a5d40dc7b7c72b4f38b06da8708ed0a6c8b8638a6
2025-03-19 13:49:34 +00:00
1fc7844d40
Teach testrunner.tcl to distinguish Cygwin from Windows. This gets it running but then all downstream tests fail, at least in part because of the build target name discrepancies between the platform which requires .exe and those which do not.
...
FossilOrigin-Name: 2861788e479aea12354f2d34d8e6a2706d193642674ef1f4f852f24c877e0140
2025-03-11 11:24:52 +00:00
e1e822ca23
Teach testrunner how to run local binaries which have a .exe extension. Also teach it to recognized mingw environments (but actually building the test programs there currently doesn't work).
...
FossilOrigin-Name: e2bd23f251359e7a818c4cfacf114aa9fd8c0a9a1cb802654e96fad67b505508
2025-02-26 19:22:52 +00:00
94c42564f3
Test testrunner.tcl to identify an msys environment.
...
FossilOrigin-Name: e9f777ceda9fca51c37caca0afeadfc3f99b86e945496f52c71adcc520dcee98
2025-02-26 18:31:42 +00:00
9c8235b1bd
Fix minor problems in testrunner.tcl that pop up when the command-line arguments
...
are such that no tests are run.
FossilOrigin-Name: 321ded32f67550e964cd64d61aa0cbc0029ce2fdee4588a46b01dbb5aa87150b
2024-11-18 13:29:16 +00:00
2139c35da0
Ensure that the one-line running status output from testrunner.tcl does not
...
exceed the 80-char line width of a standard terminal.
FossilOrigin-Name: 1eae0f9ed3c4530d117a8efc4adaf0b2cc422083f0407830a1a6d2a5b970fa1a
2024-11-05 11:59:45 +00:00
bce0d04c98
In testrunner.tcl, only show the ETC on the status line if it will fit within
...
the 80-character line limit.
FossilOrigin-Name: 45a3213d23f4691732ba2eb54d440355ce5757aad4cec8eb92f53b4bd7e7f5cd
2024-10-31 18:29:55 +00:00
0064c43a88
Improvements to the PATTERN option of the "testrunner.tcl joblist" command.
...
The pattern match is case insensitive and applies to the state and displaytype
in addition to displayname.
FossilOrigin-Name: 63a8f60f98115d4ee6e40de3ae775770aba76e19df6ddde68bccb7967b0200dc
2024-10-27 10:48:06 +00:00
f896e2bd5b
Add the "halt" command to testrunner.tcl
...
FossilOrigin-Name: 2cc25d5dbbc729f3b6deb8c6a45b975c535b4d20ad240e4ec0c85b3733a170ed
2024-10-22 20:09:18 +00:00
0d7ede8d1c
In testrunner.tcl: Better estimates for ETC (Estimated Time to Completion).
...
Show the ETC even for the single-line status reports.
FossilOrigin-Name: 2a2f5f4e378338951cc2030ffbacd678e6a6eab142e39ee632c61be42345b092
2024-10-21 22:45:59 +00:00
e2f294a1e5
Add a case to testrunner.tcl so that mdevtest can run on openbsd.
...
FossilOrigin-Name: e904b37fb2621e6bd5e761f3ecb75adb34350f2d1d7b229e655e74bc6a2f5321
2024-10-21 14:24:27 +00:00
c43d16d69a
In testrunner.tcl, correctly capture and display information about the
...
system under test: hostname, OS, pointer size, byte-order.
FossilOrigin-Name: ec75cfc5d4b69e4aed64d17748cac15cd62a759a1cbe7feaa4580ad8346b0b95
2024-09-06 13:13:25 +00:00
dc181d41d5
Fix a testrunner status reporting problem introduced by [aa5f10f21dbfb24e].
...
FossilOrigin-Name: d20c65c3b4256a662ebf7ed024b7b7adaceca90358f58111dc645da322000794
2024-09-05 23:40:13 +00:00
4987c5c3c2
Testrunner.tcl enhancements: (1) Attempt to build the SQLite tcl extension
...
if it is not already available. (2) testrunner target "devtest" is added as
an alias for "mdevtest". (3) Try to keep summary information at the end of
a test below 80-characters per line. (4) Update the Makefile.in so that
the "clean" target removes the tcl extension built by item 1 above.
FossilOrigin-Name: aa5f10f21dbfb24ee54ca96bfb7b013ae29e26fec05b80681f19cc63d9face49
2024-09-05 15:28:15 +00:00
21e7816e65
Enhance testrunner.tcl to keep track of the platform and SQLite version
...
as reported by individual test cases and to report that information in the
summary at the end of each test run.
FossilOrigin-Name: 80ebb7c7e686cd936ac834f2258f585a7004762593e0bc859ecd75d6fb0badfd
2024-09-05 12:06:45 +00:00
6aedd6d9fc
Faster implementation of the aggregate_test_counts procedure inside of
...
testrunner.tcl.
FossilOrigin-Name: a01d869520329fb9e786cdc65f359785a95d19d289e4c6b844c758d6e5385aaf
2024-09-05 11:46:43 +00:00
109025c0a4
Put a proper shebang at the start of the testrunner.tcl script and make the
...
script executable.
FossilOrigin-Name: c0c6e9abebf76358625f30a179658319b260baba6eded2a4c5ad356143e36f97
2024-09-04 13:42:19 +00:00
c81ab76cd9
Add the "joblist" command to testrunner.tcl
...
FossilOrigin-Name: f64469f4806d4d5d7103c171a37a542c7aab9db09a226bccd8411e9ccd55353d
2024-09-04 11:22:47 +00:00
fd486258a0
Enhance the "errors" command in testrunner.tcl so that it accepts the
...
"-s" or "--summary" argument to see a list of failed jobs, and so that
an additional argument is a GLOB pattern that restricts the output to
jobs whose names match that pattern.
FossilOrigin-Name: dcbebe30f594a99e23b5ccd8d199b92118204a3e52e75c78d98c394601252e81
2024-09-03 16:04:34 +00:00
06638117bb
In testrunner.tcl, allow setting njob to zero, which causes no new jobs
...
to be launched and for the process to shut down once all current jobs are
completed.
FossilOrigin-Name: 0ef65fd4ba17def4c13986365b3af300c4024725af4bc314845d1af8be568ab4
2024-09-03 10:53:32 +00:00
11397ce104
In the testrunner.tcl status report, try to provide a rough estimate of
...
the time remaining.
FossilOrigin-Name: 90bc616d20e8c247691c45de2a28d41c1632b21152dc34253eefc179a90ab31f
2024-08-30 01:11:19 +00:00
3341f5ce85
Minor wording change on testrunner output.
...
FossilOrigin-Name: 115e68597957a9ee80de7f554ee3768c992051687c8f07eaf3a9fcaab7efd44e
2024-08-28 13:34:20 +00:00
4f2f91bf08
Further improvements to the status display in order to maximum the amount of
...
information shown on the limited screen realestate available when running
via "watch" or in continuous monitoring mode. All details are still shown
when doing a stand-alone "status".
FossilOrigin-Name: 557a2c65f5ae1d8cb1858c17da0fe64fbfba83d787cfb7f2f21582960a8b9eac
2024-08-28 11:03:58 +00:00
242e064978
Only limit the number of failures reported by testrunner status if using
...
VT100 cursor movement. For a straight-up "status" command, show everything.
FossilOrigin-Name: 791237e0f4b67f0197ae11e966554edeade0c443289d2ade00470d23bf5e8ec8
2024-08-28 10:36:18 +00:00
4a90f81a6d
In the testrunner status display, limit the number of reported failed jobs
...
to avoid overflowing the terminal when there are many failures.
FossilOrigin-Name: ffeaa4d5d73871cbdf1ef70b9845d921ebdb96e964d232661a5048cab7d744ed
2024-08-28 10:25:44 +00:00
b5b4ca2a11
Changes to the way "release" tests were constructed in [d03d35eebaf82709]
...
were incorrect. Fixed here. Also enhance --explain to chose permutation
configurations and so that PATTERNS on the the command line can match
against permutation configurations.
FossilOrigin-Name: 1f962ffc9e956a9d6311ed9c02fea13c0fe056283ee2c650a2312a258808b3b3
2024-08-27 22:28:03 +00:00
c3c645ef46
Improvements to --status overwrite in testrunner.
...
FossilOrigin-Name: 26372762ab1451eb95d29367f5b2534bde616a37e5e757e72ba2cfe42571d180
2024-08-27 21:44:45 +00:00
1a7cfbebb0
Fix typo in the previous check-in. Ensure that --status header lines do not
...
overflow and wrap.
FossilOrigin-Name: 2eaea67495fd9961b7145d829789889bf1d8fa7a834d40d1be43e7c74c233bc2
2024-08-27 19:48:19 +00:00
db05c66a16
In testrunner, when doing the --status updates to the screen, be sure to
...
overwrite dead text at the end of each line with spaces.
FossilOrigin-Name: 7c195f132cb48ee44124eb4af3532937493429f7c45f6ac611f7a9b128799ad7
2024-08-27 19:43:21 +00:00
71475478a1
Further improvement to the status display in testrunner.tcl. Show the
...
number of errors and the number of test cases on the status summary.
FossilOrigin-Name: dbc1ac8692feccde271eaca78f2833c8bb7bf620088c8f975692d6ae2c8ba53a
2024-08-27 19:28:21 +00:00
d9bc2bb638
Improvements to testrunner.tcl: Show elapse time in MM:SS or HH:MM:SS.
...
Keep track of the total number of tests and the total number of errors
and report those values on a summary line at the end.
FossilOrigin-Name: 40b232924c973f8d94605946fff75acbe120d90634a2eaf38bd31649e9f1390b
2024-08-27 19:17:29 +00:00
dac0d0b310
Updates to testrunner: (1) Omit all testing of User-Auth.
...
(2) Automatically add the "*" wildcard before and after all pattern
arguments. (3) Build the sqlite3 CLI for release tests.
FossilOrigin-Name: d03d35eebaf82709414c87cfa6abc9d2baf8d7e64c2627bad0fd5bbda3e78d60
2024-08-27 17:38:26 +00:00
4a36d30493
Add the --status option to the test runs using testrunner.tcl. Only works
...
on Unix platforms. The "exec" command appears to interfere with VT100
escape codes on windows.
FossilOrigin-Name: 94015cda4ceb4292ceceadb951fe5d9cb3e4e20403719b7254ad094a5b749ab3
2024-08-03 15:55:25 +00:00
b2d590c5f9
Add the "-d SECS" option to the "status" command of testrunner.tcl
...
FossilOrigin-Name: 2151ea31632f7aa363c7d4b7205e1174a45d710783a8efad8176049d356cfc70
2024-08-02 17:36:34 +00:00
a51de65758
Include a test case summary in the output of "errors" in testrunner.tcl.
...
FossilOrigin-Name: 173df1478e89996126e172656e35da8026d4ef145b2341ef56213f00ade14f48
2024-08-01 14:43:27 +00:00
5cc7f4b580
Add the "errors" command to testrunner.tcl.
...
FossilOrigin-Name: 03b7f99229cfcffde024915e6812d75ae11318db9dc93a00838e4588a79b1243
2024-08-01 14:16:36 +00:00
b030bc6980
testrunner.db should be in WAL mode for improved concurrency while running
...
"watch sqlite3 test/testrunner.tcl status".
FossilOrigin-Name: f23ad0979865f3c11a3dbd0598ee7c4e2eabeaf1863d000fc0242e18890a4a11
2024-07-31 22:26:04 +00:00
d888e79bb8
Fix typo in testrunner.tcl for the "status" command that was causing it
...
to fail if any errors were seen.
FossilOrigin-Name: 8327c5ed535c57c4082211a50bfb1c32ea5d5fe1cf71f37798d8b58e88948ac2
2024-07-31 22:17:26 +00:00
f15c80d80f
Patch to testrunner.tcl so that it works with Tcl9's pedantic UTF-8 checking.
...
FossilOrigin-Name: 8d4a23d32dc16b115153e6e100ba3ba15273c3e37310e8a1b4bace885c5383a8
2024-07-31 11:12:13 +00:00
