Before this patch, the CMake build did not allow to disable static
libssh2 library while also building tests.
This patch removes this constraint, and makes this combination possible.
In this case the 3 (at the moment) tests that require a static libssh2
library, are skipped from the build and test runs.
Cherry-picked from #1036
Required for tests using libssh2 internals. These are the ones
requiring the libssh2 _static_ lib.
Before this patch, `src` and `tests` declared the `session` structure
differently, due to extra struct members added with the `LIBSSH2DEBUG`
macro set. But, the macro was only set for `src` when using CMake. At
runtime this caused struct members to be at different offsets between
lib and test code, resulting in the test failures below.
Due to another bug in the affected test, these failures did not reflect
in the exit code, which always returned success, so this went unnoticed
for a good while. Fixed in: 84d31d0ca7
```
Start 5: test_auth_keyboard_info_request
[...]
5: Test case 1 passed
5: Test case 2 passed
5: Test case 3: expected return code to be 0 got -1
5: Test case 4: expected last error code to be "-6" got "-38"
5: Test case 5: expected last error code to be "-6" got "-38"
5: Test case 6: expected last error code to be "-6" got "-38"
5: Test case 7: expected last error message to be "Unable to decode keyboard-interactive number of keyboard prompts" got "userauth keyboard data buffer too small to get l
5: Test case 8: expected last error code to be "-41" got "-38"
5: Test case 9: expected return code to be 0 got -1
5: Test case 10: expected return code to be 0 got -1
5: Test case 11: expected last error code to be "-6" got "-38"
5: Test case 12: expected last error message to be "Unable to decode user auth keyboard prompt echo" got "userauth keyboard data buffer too small to get length"
5: Test case 13: expected return code to be 0 got -1
5: Test case 14: expected return code to be 0 got -1
5: Test case 15: expected last error code to be "-6" got "-38"
5: Test case 16: expected last error code to be "-6" got "-38"
5: Test case 17: expected last error code to be "-6" got "-38"
5: Test case 18: expected last error code to be "-6" got "-38"
```
Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46925869/job/i9uasceu3coss0i2#L440
Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46983040/job/c3vag25c26a77lyr#L485
Cherry-picked from #1037Closes#1037
Before this patch, this test returned success even when one of its tests
failed. Fix it by returning 1 in case any of the tests fails.
This issue masked a CMake build bug with logging enabled. Subject to an
upcoming patch.
Cherry-picked from #1037
"Unity" (aka "jumbo", aka "amalgamation" builds concatenate source files
before compiling. It has these benefits for example: faster builds,
improved code optimization, cleaner code. Let's support and test this.
- enable unity builds for some existing CI builds to test this build
scenario.
- tune `UNITY_BUILD_BATCH_SIZE` size.
- disable unity build for example and test programs (they use one source
each already).
You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
Supported by CMake 3.16 and newer.
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.htmlCloses#1034
Before this patch libssh2 used a variety of solutions to pass the source
directory to tests: `FIXTURE_WORKDIR` build-time macro (cmake),
`FIXTURE_WORKDIR` envvar (unused), setting `srcdir` manually
(autotools), setting current directory (cmake), and also `builddir`
envvar (autotools) for passing current working dir to `mansyntax.sh`.
This patch reduces this to using existing `srcdir` with autotools and
setting it ourselves in CMake. This was mostly enabled by this recent
patch: 4c9ed51f96
Details:
- cmake: replace baked-in `FIXTURE_WORKDIR` macro with env.
Added in 54bef4c5da#198 (2018-03-21)
- rename `FIXTURE_WORKDIR` to `srcdir`, to match autotools.
- cmake: add missing `srcdir` for algo and sshd tests.
- session_fixture: stop `chdir()`-ing, rely on prefixing with `srcdir`.
Changing current directory should be unnecessary after
4c9ed51f96#801 (2023-02-24),
that prefixes referenced input filenames with the `srcdir` envvar.
The `srcdir` envvar was already exported by autotools, and now we're
also setting it from CMake.
- cmake: stop setting `WORKING_DIRECTORY`, rely on `srcdir` env.
`WORKING_DIRECTORY` is no longer necessary, after passing `srcdir` to
all tests, so they can find our source tree and keys/etc in it
regardless of the current directory.
Also this past commit hints that `WORKING_DIRECTORY` wasn't always
working for this purpose as expected:
"tests: Xcode doesn't obey CMake's test working directory"
Ref: 10a5cbf945
- autotools: delete explicit `srcdir` for test env.
Added in 13f8addd1b (2015-07-02)
automake documents `srcdir` as exported to the test environment:
c04c4e8856/doc/automake.texi (L9302-L9304)https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html
It's mentioned in the docs back in 1997 and got a regression test in
2012. We can safely assume it to be available without setting it
ourselves.
- autotools: delete explicit `builddir`.
Added in 13f8addd1b (2015-07-02)
It seems this wasn't necessary to make the above fix work, and
`mansyntax.sh` is able to figure out the build workdir by reading
`$PWD`. Our out-of-tree and `make distcheck` CI builds also work
without it.
Let us know if there is a scenario we're missing and needs this.
Closes#1032
`_libssh2_store_str()` and `_libssh2_store_bignum2_bytes()` accept
inputs of `size_t` max, store the size as 32-bit unsigned integer, then
store the complete input buffer.
With inputs larger than `UINT_MAX` this means the stored size is smaller
than the data that follows it.
This patch truncates the stored data to the stored size, and now returns
a boolean with false if the stored length differs from the requested
one. Also add `assert()`s for this condition.
This is still not a correct fix, as we now dump consistent, but still
truncated data which is not what the caller wants. In future steps we'll
need to update all callers that might pass large data to this function
to check the return value and handle an error, or make sure to not call
this function with more than UINT_MAX bytes of data.
Ref: c3bcdd88a4 (2010-04-17)
Ref: ed439a29bb (2022-09-29)
Closes#1025
After deleting the `bcrypt.h` check, no check remained. Restore
a `WIN32` check here to ensure WinCNG is not enabled outside Windows.
Follow-up to 1289033598
Tested-in #1032
To not force this setting for local builds where they might serve
a good purpose.
It makes our CI runs slightly faster and we don't need to track
file changes in unattended, single, CI runs.
Cherry-picked from #1031
Prefill known detection results on Windows with MinGW and MSVC, to
avoid spending time on detecting these on every cmake configuration
run.
With MinGW + clang and MSVC, this elminates all detections.
With MinGW + gcc, it reduces them to 3.
Cherry-picked from #1031
To sync up the hand-crafted config with actual detection results
by CMake and autotools. Sources compiled fine without it anyway.
Cherry-picked from #1031
`select()` detection suffered two regressions: First I accidentally
deleted it for non-Windows [1]. Then the Windows-specific setting got
missed from the generated `libssh2_config.h` after a rearrangement in
`CMakeLists.txt` files.
[1] 31fb8860db (2023-03-07)
[2] 803f19f004 (2023-04-18)
This patch restores detection. For Windows, enable it unconditionally,
not only for speed reasons, but because detection needs `ws2_32`, and
even that is broken on the x86 platform. According to the original
`cmake/SocketLibraries.cmake`, caused by a calling convention mismatch.
FWIW autotools detects it correctly.
Cherry-picked from #1031
autotools already didn't check for `bcrypt.h`, and such check is only
required for old/legacy mingw without obsolete/incomplete Windows
headers.
curl deprecated old-mingw support just recently and will delete support
in September 2023.
This patch saves some complexity and detection time by dropping this
check for CMake. Meaning that mingw-w64 is now required to compile
libssh2 when using the WinCNG backend for 32-bit builds. Other backends
and CPU platforms are not affected.
Ref: e4d5685cb5Closes#1026
While it seems to exist on mingw in theory, it's not detected as of this
writing. It also has issues, and not ready for production use:
https://stackoverflow.com/questions/1671827/poll-c-function-on-windows
On MSVC it's even less supported.
Skip checking this to save CMake detection time.
Closes#1027
Also:
- merge back `agent.h` into `agent.c` where it was earlier.
Ref: c998f79384
- introduce `HAVE_WIN32_AGENT` internal macro.
- fix two guards to exclude more code unused in UWP builds.
Follow-up to 1c1317cb76Closes#1028
Merge the loop executing multiple tests and the script that actually
launches the tests into a single script. This same script is now called
from both autotools and CMake. autotools loads the list of tests from
`Makefile.inc`, CMake passes it via the command-line. It's also possible
to call the script manually with a custom list of tests or individual
ones.
With this setup we're now launching a single sshd session for all tests,
instead of launching and killing it for each test. This did not improve
reliability of these test on CI machines, and it's easy to go back to
the previous behaviour if necessary.
Also:
- allow passing custom sshd options via `SSHD_FLAGS`.
- add `SSHD_TESTS_LIMIT_TO` to limit the number of tests to its value.
E.g. `SSHD_TESTS_LIMIT_TO=1` executes the first test only. Meant for
debugging.
- use `ssh` to test the connection (if available) instead of fixed
amount of wait. Made to also work on Windows.
- set `PermitRootLogin yes` in `sshd`, to allow running tests as root.
- show `sshd` path and version.
Cherry-picked from #1017 (the last one)
Closes#1024
Whenever possible. Due to flakiness/hangs/timeouts, keep sshd
tests disabled on Windows and macOS.
Also keep Docker tests disabled on these platforms, they do not work:
GHA Windows:
```
no matching manifest for windows/amd64 in the manifest list entries
```
GHA macOS:
```
sh: docker: command not found
```
It's not possible to run UWP and ARM64 binaries:
UWP:
```
Test #2: test_simple ......................Exit code 0xc0000135
```
Needs but doesn't find: `VCRUNTIME140_APP.dll`.
ARM64
```
D:/a/libssh2/libssh2/bld/tests/Release/test_ssh2.exe: cannot execute binary file: Exec format error
```
Cherry-picked from #1017
Instead of using hacks inside the build systems.
`SSHD` variable added to GitHub Actions is not currently used.
Added there to make it easy to experiment with these tests and
the path is non-trivial to discover. Using the Windows built-in
sshd server is another option (haven't discovered its path yet).
Cherry-picked from #1017
autotools:
- `--disable-docker-tests`
- `--disable-sshd-tests`
cmake:
- `RUN_DOCKER_TESTS`
- `RUN_SSHD_TESTS`
Update automake and ci to use this new flag and delete former logic
of relying on Windows detection and `HOST_WINDOWS`. Also fix honoring
this when running `test_read_algos.test`.
This allows to disable these individually and on per-CI/local-job basis.
To run as much tests as the env allows.
Cherry-picked from #1017
Also:
- delete an unused 64-bit option for Linux (all jobs are 64-bit).
- fix to not install libgcrypt and openssl when doing mbedTLS builds.
[ Empty lines after multiline run commands are solely to unbreak
my editor's syntax highlighting. They can be deleted in the future ]
Cherry-picked from #1017
Instead of assembling the list using `$(wildcard ...)`.
Also split off a `tests/Makefile.inc` from `tests/Makefile.am`. With its
simpler syntax, this also allows to delete some complexity from the
CMake loader.
Cherry-picked from #1017
Before this patch ssh2 and test_ssh2 returned success even if the session
failed at `libssh2_session_handshake()` or after.
This patch depends on cda41f7cb8, that fixed
running test_ssh2 on Windows via sshd_fixture.
Cherry-picked from #1017
Re-sync with autotools and v1.10.0 behavior.
This improves build times. It also allows to stop building our special
shared test target to test shared builds.
Follow-up to 4e2580628d
Cherry-picked from #1017Closes#1022
Twice. This tests are flaky and we haven't figured out why. In the
meantime use this workaround to test and log these issues, but also
ensure that CI run aren't flagged red because of it.
Also:
- kex: add debug message when hostkey `sig_verify` fails,
to help tracking WinCNG KEX failures.
- test_ssh2: also add retry logic.
I'm not quite sure this is correct. Please let me know.
- session_fixture: bump up `src_path` slots to fit retries and show
message when hitting the limit.
- session_fixture: clear `kbd_password` static variable after use.
- session_fixture: close and deinit socket after use.
- session_fixture: deinit libssh2 after use.
Ref: #804#846#979#1012#1015
Cherry-picked from #1017Closes#1023
Make sure these files get LF newlines on checkout. Before this patch
a checked out libssh2 Git repository may have used CRLF newlines in text
files, include test keys. Private keys with CRLF newlines could confuse
sshd on Windows:
```
# sshd version: 'OpenSSH_9.2, OpenSSL 1.1.1t 7 Feb 2023'
Unable to load host key "/d/a/libssh2/libssh2/tests/openssh_server/ssh_host_ed25519_key": invalid format
Unable to load host key: /d/a/libssh2/libssh2/tests/openssh_server/ssh_host_ed25519_key
```
Ref: https://github.com/libssh2/libssh2/actions/runs/4846188677/jobs/8635575847#step:6:39
Cherry-picked from #1017
Follow-up to 3c953c05d6#797
There is pending issue with wolfSSL, where encryption/decryption is not
working (both with and without this patch). Ref: #1020
Cherry-picked from #1017
