autotools already didn't check for `bcrypt.h`, and such check is only
required for old/legacy mingw without obsolete/incomplete Windows
headers.
curl deprecated old-mingw support just recently and will delete support
in September 2023.
This patch saves some complexity and detection time by dropping this
check for CMake. Meaning that mingw-w64 is now required to compile
libssh2 when using the WinCNG backend for 32-bit builds. Other backends
and CPU platforms are not affected.
Ref: e4d5685cb5Closes#1026
While it seems to exist on mingw in theory, it's not detected as of this
writing. It also has issues, and not ready for production use:
https://stackoverflow.com/questions/1671827/poll-c-function-on-windows
On MSVC it's even less supported.
Skip checking this to save CMake detection time.
Closes#1027
Also:
- merge back `agent.h` into `agent.c` where it was earlier.
Ref: c998f79384
- introduce `HAVE_WIN32_AGENT` internal macro.
- fix two guards to exclude more code unused in UWP builds.
Follow-up to 1c1317cb76Closes#1028
Merge the loop executing multiple tests and the script that actually
launches the tests into a single script. This same script is now called
from both autotools and CMake. autotools loads the list of tests from
`Makefile.inc`, CMake passes it via the command-line. It's also possible
to call the script manually with a custom list of tests or individual
ones.
With this setup we're now launching a single sshd session for all tests,
instead of launching and killing it for each test. This did not improve
reliability of these test on CI machines, and it's easy to go back to
the previous behaviour if necessary.
Also:
- allow passing custom sshd options via `SSHD_FLAGS`.
- add `SSHD_TESTS_LIMIT_TO` to limit the number of tests to its value.
E.g. `SSHD_TESTS_LIMIT_TO=1` executes the first test only. Meant for
debugging.
- use `ssh` to test the connection (if available) instead of fixed
amount of wait. Made to also work on Windows.
- set `PermitRootLogin yes` in `sshd`, to allow running tests as root.
- show `sshd` path and version.
Cherry-picked from #1017 (the last one)
Closes#1024
Whenever possible. Due to flakiness/hangs/timeouts, keep sshd
tests disabled on Windows and macOS.
Also keep Docker tests disabled on these platforms, they do not work:
GHA Windows:
```
no matching manifest for windows/amd64 in the manifest list entries
```
GHA macOS:
```
sh: docker: command not found
```
It's not possible to run UWP and ARM64 binaries:
UWP:
```
Test #2: test_simple ......................Exit code 0xc0000135
```
Needs but doesn't find: `VCRUNTIME140_APP.dll`.
ARM64
```
D:/a/libssh2/libssh2/bld/tests/Release/test_ssh2.exe: cannot execute binary file: Exec format error
```
Cherry-picked from #1017
Instead of using hacks inside the build systems.
`SSHD` variable added to GitHub Actions is not currently used.
Added there to make it easy to experiment with these tests and
the path is non-trivial to discover. Using the Windows built-in
sshd server is another option (haven't discovered its path yet).
Cherry-picked from #1017
autotools:
- `--disable-docker-tests`
- `--disable-sshd-tests`
cmake:
- `RUN_DOCKER_TESTS`
- `RUN_SSHD_TESTS`
Update automake and ci to use this new flag and delete former logic
of relying on Windows detection and `HOST_WINDOWS`. Also fix honoring
this when running `test_read_algos.test`.
This allows to disable these individually and on per-CI/local-job basis.
To run as much tests as the env allows.
Cherry-picked from #1017
Also:
- delete an unused 64-bit option for Linux (all jobs are 64-bit).
- fix to not install libgcrypt and openssl when doing mbedTLS builds.
[ Empty lines after multiline run commands are solely to unbreak
my editor's syntax highlighting. They can be deleted in the future ]
Cherry-picked from #1017
Instead of assembling the list using `$(wildcard ...)`.
Also split off a `tests/Makefile.inc` from `tests/Makefile.am`. With its
simpler syntax, this also allows to delete some complexity from the
CMake loader.
Cherry-picked from #1017
Before this patch ssh2 and test_ssh2 returned success even if the session
failed at `libssh2_session_handshake()` or after.
This patch depends on cda41f7cb8, that fixed
running test_ssh2 on Windows via sshd_fixture.
Cherry-picked from #1017
Re-sync with autotools and v1.10.0 behavior.
This improves build times. It also allows to stop building our special
shared test target to test shared builds.
Follow-up to 4e2580628d
Cherry-picked from #1017Closes#1022
Twice. This tests are flaky and we haven't figured out why. In the
meantime use this workaround to test and log these issues, but also
ensure that CI run aren't flagged red because of it.
Also:
- kex: add debug message when hostkey `sig_verify` fails,
to help tracking WinCNG KEX failures.
- test_ssh2: also add retry logic.
I'm not quite sure this is correct. Please let me know.
- session_fixture: bump up `src_path` slots to fit retries and show
message when hitting the limit.
- session_fixture: clear `kbd_password` static variable after use.
- session_fixture: close and deinit socket after use.
- session_fixture: deinit libssh2 after use.
Ref: #804#846#979#1012#1015
Cherry-picked from #1017Closes#1023
Make sure these files get LF newlines on checkout. Before this patch
a checked out libssh2 Git repository may have used CRLF newlines in text
files, include test keys. Private keys with CRLF newlines could confuse
sshd on Windows:
```
# sshd version: 'OpenSSH_9.2, OpenSSL 1.1.1t 7 Feb 2023'
Unable to load host key "/d/a/libssh2/libssh2/tests/openssh_server/ssh_host_ed25519_key": invalid format
Unable to load host key: /d/a/libssh2/libssh2/tests/openssh_server/ssh_host_ed25519_key
```
Ref: https://github.com/libssh2/libssh2/actions/runs/4846188677/jobs/8635575847#step:6:39
Cherry-picked from #1017
Follow-up to 3c953c05d6#797
There is pending issue with wolfSSL, where encryption/decryption is not
working (both with and without this patch). Ref: #1020
Cherry-picked from #1017
- update `AC_HELP_STRING' to 'AS_HELP_STRING`:
```
configure.ac:[...]: warning: The macro `AC_HELP_STRING' is obsolete.
```
"AC_HELP_STRING is deprecated in 2.70+ and I believe AS_HELP_STRING works
already since 2.59 so bump the minimum required version to that."
Ref: a59f046116
- simplify to avoid:
```
src/Makefile.inc:48: warning: variable 'EXTRA_DIST_SOURCES' is defined but no program or
src/Makefile.inc:48: library has 'DIST' as canonical name (possible typo)
```
Regression from 2c18b6fc8d
- `AC_TRY_LINK`/`AC_TRY_COMPILE`:
```
configure.ac:335: warning: The macro `AC_TRY_COMPILE' is obsolete.
configure.ac:335: warning: The macro `AC_TRY_LINK' is obsolete.
```
- `libtool`-related ones:
```
configure.ac:70: warning: The macro `AC_LIBTOOL_WIN32_DLL' is obsolete.
configure.ac:70: warning: AC_LIBTOOL_WIN32_DLL: Remove this warning and the call to _LT_SET_OPTION when you
configure.ac:70: put the 'win32-dll' option into LT_INIT's first parameter.
configure.ac:71: warning: The macro `AC_PROG_LIBTOOL' is obsolete.
```
Using code copied from curl:
9ce7eee070/m4/xc-lt-iface.m4 (L157-L163)
- delete commented and obsolete `AC_HEADER_STDC`.
- formatting.
Most cherry-picked from `autoupdate` updates.
Cherry-picked from #1017Closes#1021
Currently libssh2 cannot read huge directory listings when the package
size of `LIBSSH2_SFTP_PACKET_MAXLEN` (256KB) is hit. For example AWS
always sends a single package with all files of a directory, no matter
how big it is: https://freefilesync.org/forum/viewtopic.php?t=10020
Package size is probably around 7MB in this case!
`LIBSSH2_SFTP_PACKET_MAXLEN` is a good idea in general, but there
doesn't seem to be a one size fits all. While almost all(?) SFTP
responses come in very small packages, I believe the `SSH_FXP_READDIR`
request should be exempted.
The proposed patch, enhances the package size reading to include parsing
the full SFTP packet header. And in case a package is of type
`SSH_FXP_NAME` and matches an expected `readdir_request_id`, it does not
fail if `LIBSSH2_SFTP_PACKET_MAXLEN` is hit. The chances of accidentally
hiding data-corruption are pretty non-existent, because both SFTP
`request_id` and packet type must match. No change in behavior
otherwise.
Best, Zenju
Previous discussion: #268#269
With the above changes, the `LIBSSH2_SFTP_PACKET_MAXLEN` value could
(and should?) probably be set back to a small number again.
Integration-patches-by: Viktor Szakats
Add macOS CI jobs, both cmake and autotools for all supported crypto
backends (except BoringSSL), with debug, zlib enabled. Without running
tests. It also introduces OpenSSL 1.1 into the CI with a non-MSVC
compiler.
Credits to curl's `macos.yml`, that I used as a base.
Fix these issues uncovered by the new tests:
- openssl: fix warning when built with wolfSSL, or OpenSSL 1.1 and
earlier. CI missed it because apparently the only OpenSSL 1.1 test
we had used MSVC, which did not complain.
```
../src/openssl.c:3852:19: error: variable 'sslError' set but not used [-Werror,-Wunused-but-set-variable]
unsigned long sslError;
^
```
Regression from 097c8f0dae
- pem: add hack to build without MD5 crypto-backend support.
The Homebrew wolfSSL build comes with MD5 support disabled. We can
expect this becoming the norm. FIPS also requires MD5 disabled.
We deleted the same hack from `hostkey.c` a month ago:
ad6aae302a
A better fix would be to guard the MD5 logic with our `LIBSSH2_MD5`
macro.
```
pem.c:214:32: error: use of undeclared identifier 'MD5_DIGEST_LENGTH'; did you mean 'SHA_DIGEST_LENGTH'?
unsigned char secret[2*MD5_DIGEST_LENGTH];
^~~~~~~~~~~~~~~~~
SHA_DIGEST_LENGTH
```
Regression from 386e012292
- `configure.ac`: add crypto libs late.
Fix it by adding crypto libs to `LIBS` at the end of the configuration
process.
Otherwise `configure` links crypto libs while doing feature tests,
which can cause unwanted detections. For example LibreSSL publishes
the function `explicit_bzero()`, which masks the system alternative,
e.g. `memset_s()` on macOS. Then when trying to compile libssh2, its
declaration is missing:
```
bcrypt_pbkdf.c:93:5: error: implicit declaration of function 'explicit_bzero' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
_libssh2_explicit_zero(ciphertext, sizeof(ciphertext));
^
../src/misc.h:50:43: note: expanded from macro '_libssh2_explicit_zero'
^
```
Regression from 4f0f4bff5a
- cmake: fix to list our own include directory before the crypto libs',
when building tests.
Otherwise a global crypto header path, such as `/usr/local/include`,
containing an external `libssh2.h` of a different version, could cause
weird errors:
```
cc -DHAVE_CONFIG_H -DLIBSSH2_LIBGCRYPT \
-I../src -I../../src -I/usr/local/include -I[...]/libssh2/include \
-g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk \
-mmacosx-version-min=12.6 -MD -MT \
tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o \
-MF CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o.d \
-o CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o -c \
[...]/libssh2/tests/test_aa_warmup.c
```
```
[ 62%] Building C object tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o
In file included from /Users/runner/work/libssh2/libssh2/tests/test_aa_warmup.c:4:
In file included from /Users/runner/work/libssh2/libssh2/tests/runner.h:42:
In file included from /Users/runner/work/libssh2/libssh2/tests/session_fixture.h:43:
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:5: error: type name requires a specifier or qualifier
LIBSSH2_AUTHAGENT_FUNC((*authagent));
^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:30: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
LIBSSH2_AUTHAGENT_FUNC((*authagent));
^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:5: error: type name requires a specifier or qualifier
LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities));
^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities));
^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:5: error: type name requires a specifier or qualifier
LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback));
^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback));
^
6 errors generated.
```
- `tests/session_fixture.h`: delete duplicate `libssh2.h`,
`libssh2_priv.h` already includes it.
Follow-up to a683133dfe
CI logs with these errors:
https://github.com/libssh2/libssh2/actions/runs/4824079094https://github.com/libssh2/libssh2/actions/runs/4824270819
curl's `macos.yml`: da2470de96/.github/workflows/macos.yml
Tidying-up while here:
- tests/session_fixture.h: delete duplicate `libssh2.h`.
`libssh2_priv.h` includes it already.
Follow-up to a683133dfe
- ci.yml: yamllint warnings and formatting.
- ci.yml: msvc section formatting and step-naming sync with macOS.
Follow-up to f4a4c05dc3
- ci.yml: enable `--enable-werror` for msys2 jobs.
Follow-up to 71cae949d5
- appveyor.yml: show OpenSSL versions, link to image content.
Closes#1013
