Re-sync with autotools and v1.10.0 behavior.
This improves build times. It also allows to stop building our special
shared test target to test shared builds.
Follow-up to 4e2580628d
Cherry-picked from #1017Closes#1022
Twice. This tests are flaky and we haven't figured out why. In the
meantime use this workaround to test and log these issues, but also
ensure that CI run aren't flagged red because of it.
Also:
- kex: add debug message when hostkey `sig_verify` fails,
to help tracking WinCNG KEX failures.
- test_ssh2: also add retry logic.
I'm not quite sure this is correct. Please let me know.
- session_fixture: bump up `src_path` slots to fit retries and show
message when hitting the limit.
- session_fixture: clear `kbd_password` static variable after use.
- session_fixture: close and deinit socket after use.
- session_fixture: deinit libssh2 after use.
Ref: #804#846#979#1012#1015
Cherry-picked from #1017Closes#1023
Make sure these files get LF newlines on checkout. Before this patch
a checked out libssh2 Git repository may have used CRLF newlines in text
files, include test keys. Private keys with CRLF newlines could confuse
sshd on Windows:
```
# sshd version: 'OpenSSH_9.2, OpenSSL 1.1.1t 7 Feb 2023'
Unable to load host key "/d/a/libssh2/libssh2/tests/openssh_server/ssh_host_ed25519_key": invalid format
Unable to load host key: /d/a/libssh2/libssh2/tests/openssh_server/ssh_host_ed25519_key
```
Ref: https://github.com/libssh2/libssh2/actions/runs/4846188677/jobs/8635575847#step:6:39
Cherry-picked from #1017
Follow-up to 3c953c05d6#797
There is pending issue with wolfSSL, where encryption/decryption is not
working (both with and without this patch). Ref: #1020
Cherry-picked from #1017
- update `AC_HELP_STRING' to 'AS_HELP_STRING`:
```
configure.ac:[...]: warning: The macro `AC_HELP_STRING' is obsolete.
```
"AC_HELP_STRING is deprecated in 2.70+ and I believe AS_HELP_STRING works
already since 2.59 so bump the minimum required version to that."
Ref: a59f046116
- simplify to avoid:
```
src/Makefile.inc:48: warning: variable 'EXTRA_DIST_SOURCES' is defined but no program or
src/Makefile.inc:48: library has 'DIST' as canonical name (possible typo)
```
Regression from 2c18b6fc8d
- `AC_TRY_LINK`/`AC_TRY_COMPILE`:
```
configure.ac:335: warning: The macro `AC_TRY_COMPILE' is obsolete.
configure.ac:335: warning: The macro `AC_TRY_LINK' is obsolete.
```
- `libtool`-related ones:
```
configure.ac:70: warning: The macro `AC_LIBTOOL_WIN32_DLL' is obsolete.
configure.ac:70: warning: AC_LIBTOOL_WIN32_DLL: Remove this warning and the call to _LT_SET_OPTION when you
configure.ac:70: put the 'win32-dll' option into LT_INIT's first parameter.
configure.ac:71: warning: The macro `AC_PROG_LIBTOOL' is obsolete.
```
Using code copied from curl:
9ce7eee070/m4/xc-lt-iface.m4 (L157-L163)
- delete commented and obsolete `AC_HEADER_STDC`.
- formatting.
Most cherry-picked from `autoupdate` updates.
Cherry-picked from #1017Closes#1021
Currently libssh2 cannot read huge directory listings when the package
size of `LIBSSH2_SFTP_PACKET_MAXLEN` (256KB) is hit. For example AWS
always sends a single package with all files of a directory, no matter
how big it is: https://freefilesync.org/forum/viewtopic.php?t=10020
Package size is probably around 7MB in this case!
`LIBSSH2_SFTP_PACKET_MAXLEN` is a good idea in general, but there
doesn't seem to be a one size fits all. While almost all(?) SFTP
responses come in very small packages, I believe the `SSH_FXP_READDIR`
request should be exempted.
The proposed patch, enhances the package size reading to include parsing
the full SFTP packet header. And in case a package is of type
`SSH_FXP_NAME` and matches an expected `readdir_request_id`, it does not
fail if `LIBSSH2_SFTP_PACKET_MAXLEN` is hit. The chances of accidentally
hiding data-corruption are pretty non-existent, because both SFTP
`request_id` and packet type must match. No change in behavior
otherwise.
Best, Zenju
Previous discussion: #268#269
With the above changes, the `LIBSSH2_SFTP_PACKET_MAXLEN` value could
(and should?) probably be set back to a small number again.
Integration-patches-by: Viktor Szakats
Add macOS CI jobs, both cmake and autotools for all supported crypto
backends (except BoringSSL), with debug, zlib enabled. Without running
tests. It also introduces OpenSSL 1.1 into the CI with a non-MSVC
compiler.
Credits to curl's `macos.yml`, that I used as a base.
Fix these issues uncovered by the new tests:
- openssl: fix warning when built with wolfSSL, or OpenSSL 1.1 and
earlier. CI missed it because apparently the only OpenSSL 1.1 test
we had used MSVC, which did not complain.
```
../src/openssl.c:3852:19: error: variable 'sslError' set but not used [-Werror,-Wunused-but-set-variable]
unsigned long sslError;
^
```
Regression from 097c8f0dae
- pem: add hack to build without MD5 crypto-backend support.
The Homebrew wolfSSL build comes with MD5 support disabled. We can
expect this becoming the norm. FIPS also requires MD5 disabled.
We deleted the same hack from `hostkey.c` a month ago:
ad6aae302a
A better fix would be to guard the MD5 logic with our `LIBSSH2_MD5`
macro.
```
pem.c:214:32: error: use of undeclared identifier 'MD5_DIGEST_LENGTH'; did you mean 'SHA_DIGEST_LENGTH'?
unsigned char secret[2*MD5_DIGEST_LENGTH];
^~~~~~~~~~~~~~~~~
SHA_DIGEST_LENGTH
```
Regression from 386e012292
- `configure.ac`: add crypto libs late.
Fix it by adding crypto libs to `LIBS` at the end of the configuration
process.
Otherwise `configure` links crypto libs while doing feature tests,
which can cause unwanted detections. For example LibreSSL publishes
the function `explicit_bzero()`, which masks the system alternative,
e.g. `memset_s()` on macOS. Then when trying to compile libssh2, its
declaration is missing:
```
bcrypt_pbkdf.c:93:5: error: implicit declaration of function 'explicit_bzero' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
_libssh2_explicit_zero(ciphertext, sizeof(ciphertext));
^
../src/misc.h:50:43: note: expanded from macro '_libssh2_explicit_zero'
^
```
Regression from 4f0f4bff5a
- cmake: fix to list our own include directory before the crypto libs',
when building tests.
Otherwise a global crypto header path, such as `/usr/local/include`,
containing an external `libssh2.h` of a different version, could cause
weird errors:
```
cc -DHAVE_CONFIG_H -DLIBSSH2_LIBGCRYPT \
-I../src -I../../src -I/usr/local/include -I[...]/libssh2/include \
-g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk \
-mmacosx-version-min=12.6 -MD -MT \
tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o \
-MF CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o.d \
-o CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o -c \
[...]/libssh2/tests/test_aa_warmup.c
```
```
[ 62%] Building C object tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o
In file included from /Users/runner/work/libssh2/libssh2/tests/test_aa_warmup.c:4:
In file included from /Users/runner/work/libssh2/libssh2/tests/runner.h:42:
In file included from /Users/runner/work/libssh2/libssh2/tests/session_fixture.h:43:
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:5: error: type name requires a specifier or qualifier
LIBSSH2_AUTHAGENT_FUNC((*authagent));
^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:30: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
LIBSSH2_AUTHAGENT_FUNC((*authagent));
^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:5: error: type name requires a specifier or qualifier
LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities));
^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities));
^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:5: error: type name requires a specifier or qualifier
LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback));
^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback));
^
6 errors generated.
```
- `tests/session_fixture.h`: delete duplicate `libssh2.h`,
`libssh2_priv.h` already includes it.
Follow-up to a683133dfe
CI logs with these errors:
https://github.com/libssh2/libssh2/actions/runs/4824079094https://github.com/libssh2/libssh2/actions/runs/4824270819
curl's `macos.yml`: da2470de96/.github/workflows/macos.yml
Tidying-up while here:
- tests/session_fixture.h: delete duplicate `libssh2.h`.
`libssh2_priv.h` includes it already.
Follow-up to a683133dfe
- ci.yml: yamllint warnings and formatting.
- ci.yml: msvc section formatting and step-naming sync with macOS.
Follow-up to f4a4c05dc3
- ci.yml: enable `--enable-werror` for msys2 jobs.
Follow-up to 71cae949d5
- appveyor.yml: show OpenSSL versions, link to image content.
Closes#1013
- drop unused or duplicate C headers.
- add missing ones (that worked by chance).
(`string.h`, `stdlib.h`)
- mention the functions that need certain headers.
- move some headers from crypto header to crypto C source.
- reorder headers in some places.
- simplify the #if tree for `sys/select.h` in `libssh2_priv.h`.
- move scp-specific macros next to their header to `scp.c`
Follow-up to 5db836b2a8Closes#999
- add MSVC jobs to GitHub Actions. They are similar to the 'Build-only'
jobs we have on AppVeyor CI, though only the ARM64 Windows one is
identical. Major disadvantage is that we don't run tests here. Major
advantage is they only take a few minutes to complete, compared to
an hour on AppVeyor, so WinCNG build results now appear quicker.
Docker tests might be possible, but my light attempts failed.
Finding ZLIB also failed, so we still miss an MSVC test with it.
Tool versions as of now: Server 2022, VS2022, OpenSSL 1.1.1
- add UWP builds for both ARM64 and x64. This hasn't been CI tested
before.
(We could probably enable UWP on AppVeyor CI as well.
I haven't tried.)
- fix two uncovered UWP issues in tests.
- rename internal macro `LIBSSH2_WINDOWS_APP` to `LIBSSH2_WINDOWS_UWP`.
Follow-up to 2addafb77b
- fold long lines and quote truthy values in `.github/workflows/ci.yml`.
Closes#1010
If no `FIXTURE_WORKDIR` macro or envvar is present to set the cwd,
avoid querying the cwd and then calling chdir with the result.
Ref: 54bef4c5da (patch)
Ref: 10a5cbf945 (individual commit)
Closes#1009
Currently SFTP's init will busy loop waiting for the channel to close,
even if the underlying transport returns EAGAIN. While this works for
sockets, it might not work out if you have a different transport that
needs to do some additional processing on the side.
Integration-patches-by: Viktor Szakats
- simplify `.TH` headers.
- delete empty lines before sections.
- update template with an `AVAILABILITY` section.
Left libssh2 version number in the `.TH` header for entries without an
`AVAILABILITY` section, or where there was a different version number
there.
Due to a typo in the `HAVE_*` macro, this header was never included.
A comment suggests that `socklen_t` is not defined on VMS and defines it
manually. This symbol is usually in `sys/socket.h`, so the typo may have
been the reason for it to be missing.
Closes#1007
- add #included C files to `EXTRA_DIST`.
Regression from 4f0f4bff5a
- fix `tests/sshd_fixture.sh` to not write into the test dir, by using
a pre-assembled `TrustedUserCAKeys` file. Update `Dockerfile` too to
use this.
Regression from a459a25302
Also update `tests/sshd_fixture.sh` to use
`openssh_server/authorized_keys` like `Dockerfile` does. And a few more
cosmetic updates.
Closes#1006
Unless I'm missing something, it looks like `libssh2.h` has been using
`libssh2_int64_t` unconditionally since at least 2010-04-17 when
`libssh2_scp_send64()` landed via commit
be9ee7095e.
This makes it redundant to detect `HAVE_LONGLONG` to fallback to a
32-bit `scpRecv_size` in `libssh2_priv.h`. Then deal with possible
combinations of this flag and `strtoll()` options, which was
error-prone.
Instead, assume in `libssh2_priv.h` that we have `libssh2_int64_t`, and
use it always.
For MSVC, this means `_MSC_VER` `1310` (from year 2003) is now
required. Based on the above, this was already so before this patch.
If there happens to be no 64-bit `strtoll()` detected, fall back to the
32-bit `strtol()` (this should never happen with MSVC, and probably
neither with any other reasonably modern toolchain.)
Also make sure to set `HAVE_STRTOI64` for older, non-CMake, MSVC builds
(e.g. `Makefile.mk` or `NMakefile` ones).
Closes#1002
- openssl: fix potentially missing `ERR_*` constants by including
`openssl/err.h`. This could happen with recent version of Xcode
or when building against OpenSSL built with the `OPENSSL_NO_ENGINE`
option.
Regression from 097c8f0dae (#789)
- channel: fix an issue that would corrupt the data stream when
attempting to initialize the agent in non-blocking mode, as it is
necessary to propagate the `EAGAIN` signal upstream when the transport
returns `EAGAIN`.
Regression from bc4e619e76 (#752)
- packet: the current code does not set the state machine upon reaching
this point which means that if the code is suspended due to the
transport returning an `EAGAIN`, this will re-initialize the structure
every time.
The issue is that this keeps assigning a new channel-id downstream,
which does not match the initial channel-id that is initially
generated, causing a lookup later to fail as there is no matching
channel.
Regression from bc4e619e76 (#752)
Simplify the way we handle `gettimeofday()` fallback for platforms
without native support or without any support. Make it similar to
how we handle `snprintf()`.
In case of no native `gettimeofday()` support and a non-Windows
platform, our local fallback returns zero in `tv_usec` and `tv_sec`,
ending up with a zero `timeout_remaining` in `session.c`, same as
before this patch.
Also:
- drop unused `sys/time.h` headers.
- fix our fallback code to compile with any Windows compilers
(not just MSVC)
- delete unnecessary casts.
Closes#1001
Noticed this when libasan started kicking out errors when sending in
MACs preferences that were not supported yet.
Reported-by: fourierules on github
Fixes#611Closes#1000
