Noticed this when libasan started kicking out errors when sending in
MACs preferences that were not supported yet.
Reported-by: fourierules on github
Fixes#611Closes#1000
TL;DR: Sync test builds between autotools and CMake. Sync sshd
configuration between Docker and non-Docker fixtures. Bump up
sshd_config for recent OpenSSH releases.
This also opens up the path to have non-Docker tests that use a
local sshd process. Though sshd is practically unusable on Windows
CI machines out of the box, so this will need further efforts.
Details:
- cmake: run sshd fixture test just like autotool did already.
- sync tests and their order between autotools and CMake.
It makes `test_aa_warmup` the first test with both.
- cmake: load test lists from `Makefile.am`.
Needed to update the loader to throw away certain lines to keep the
converted output conform CMake syntax. Using regexp might be an
alternative way of doing this, but couldn't make it work.
- cmake: use the official way to configure test environment variables.
Switch to syntax that's extendable.
- cmake: allow to run the same test both under Docker and sshd fixture.
Useful for testing the sshd fixture runner, or how the same test
behaves in each fixture.
- update test fixture to read the username from `USER` envvar instead of
using the Dockfile-specific hardwired one, when running outside Docker.
- rework `ssh2.sh` into `sshd_fixture.sh`, to:
- allow running any tests (not just `test_ssh2`).
- configure Docker tests for running outside Docker.
- fixup `SSHD` path when running on Windows (e.g. in AppVeyor CI).
Fixes: `sshd re-exec requires execution with an absolute path`
- allow overriding `PUBKEY` and `PRIVKEY` envvars.
- allow overriding `ssh_config` via `SSHD_FIXTURE_CONFIG`.
- prepare support for running multiple tests via sshd_fixture.
Add a TAP runner for autotools and extend CMake logic. The TAP runner
loads the test list from `Makefile.am`.
Notice however that on Windows, `sshd_fixture.sh` is very flaky with
GitHub Actions. And consistently broken for subsequent tests in
AppVeyor CI:
'libssh2_session_handshake failed (-43): Failed getting banner'
Another way to try is a single sshd instance serving all tests.
For CMake this would probably mean using an external script.
- ed25519 test keys were identical for auth and host. Regenerate the
auth keypair to make them distinct.
- sync the sshd environment between Docker and sshd_fixture.
- use common via `openssh_server/sshd_config`.
- accept same auth keys.
- offer the same host keys.
- sync TrustedUserCAKeys.
- delete now unused keypairs: `etc/host*`, `etc/user*`.
- bump up startup delay for Windows (randomly, to 5 secs, from 3).
- delete `UsePrivilegeSeparation no` to avoid deprecation warnings.
`command-line line 0: Deprecated option UsePrivilegeSeparation`
- delete `Protocol 2` to avoid deprecation warnings.
It has been the default since OpenSSH 3.0 (2001-11-06).
- delete `StrictModes no` (CI tests work without it, Docker tests
never used it).
- bump `Dockerfile` base image to `testing-slim` (from `bullseye-slim`).
It needed `sshd_config` updates to keep things working with
OpenSSH 9.2 (compared to bullseye's 8.4).
- replace `ChallengeResponseAuthentication` alias with
`KbdInteractiveAuthentication`.
The former is no longer present in default `sshd_config` since
OpenSSH 8.7 (2021-08-20). This broke the `Dockerfile` script.
The new name is documented since OpenSSH 4.9 (2008-03-31)
- add `PubkeyAcceptedKeyTypes +ssh-rsa,ssh-dss,ssh-rsa-cert-v01@openssh.com`
and `HostKeyAlgorithms +ssh-rsa`.
Original-patch-by: Eric van Gyzen (@vangyzen on github)
Fixes#691
There is a new name for `PubkeyAcceptedKeyTypes`:
`PubkeyAcceptedAlgorithms`.
It requires OpenSSH 8.5 (2021-03-03) and breaks some envs so we're
not using it just yet.
- drop `rijndael-cbc@lysator.liu.se` tests and references from config.
This is a draft alias for `aes256-cbc`. No need to test it twice.
Also this alias is no longer recognized by OpenSSH 8.5 (2021-03-03).
- update `mansyntax.sh` and `sshd_fixture.sh` to not rely on `srcdir`.
Hopefully this works with out-of-tree builds.
- fix `test_read_algos.test` to honor CRLF EOLs in their inputs
(necessary when running on Windows.)
- fix `test_read_algos.test` to honor `EXEEXT`. Might be useful when
running tests under cross-builds?
- `test_ssh2.c`:
- use libssh2 API to set blocking mode. This makes it support all
platforms.
- adapt socket open timeout logic from `openssh_fixture.c`.
Sadly this did not help fix flakiness on GHA Windows.
- tests: delete unused C headers and variable initialization.
- delete unused test files: `sshd_fixture.sh.in`, `sshdwrap`,
`etc/sshd_config`.
Ref: cf80f2f4b5
- autotools: delete stray `.c` test sources from `EXTRA_DIST` in tests.
- `tests/.gitignore`: drop two stray tests.
- autotools: fix passing `SSHD` containing space (Windows needs this).
- autotools: sort `EXTRA_DIST` in tests.
- cmake: fix to add `test_ssh2` to `TEST_TARGETS`.
- fix `authorized_key` order in `tests/gen_keys.sh`.
- silence shellcheck warning in `ci/checksrc.sh`.
- set `SSHD` for autotools on GitHub Actions Windows. [skipped]
Auto-detection doesn't work (maybe because sshd is installed via
Git for Windows and we're using MSYS2's shell.)
It enables running sshd fixture (non-Docker) tests in these jobs.
I did not include this in the final patch due to flakiness:
```
Connection to 127.0.0.1:4711 attempt #0 failed: retrying...
Connection to 127.0.0.1:4711 attempt #1 failed: retrying...
Connection to 127.0.0.1:4711 attempt #2 failed: retrying...
Failure establishing SSH session: -43
```
Can be enabled with:
`export SSHD='C:/Program Files/Git/usr/bin/sshd.exe'`
Closes#996
Make the block count customizable in `test_read` via environment
`FIXTURE_XFER_COUNT`.
Set the custom count lower than the default when running on AppVeyor.
The goal is to reduce CI roundtrip times.
Closes#995
This PR contains a series of patches that date back many years and I
believe were discussed on the mailing list, but never merged. We have
been using these in our local copy of libssh2 without issue since 2015,
if not earlier. I believe this is the full set of changes, as we tried
to use comments to mark where our copy of libssh2 differs from the
canonical version.
This also contains changes I made earlier this year, but which were not
discussed on the mailing list, to support certificates and FIDO2 keys
with agent forwarding.
Note that this is not a complete implementation of agent forwarding, as
that is outside the scope of libssh2. Clients still need to provide
their own implementation that parses ssh-agent methods after calling
libssh2_channel_read() and calls the appropriate callback messages in
libssh2. See the man page changes in this PR for more details.
Integration-patches-by: Viktor Szakats
* prefer size_t
* prefer unsigned int over u_int in public function
* add const
* docs, indent, checksrc, debug call, compiler warning fixes
We had Windows Server 2012 R2 (8.1) and Windows Server 2019 (10) before
this patch. After, we also have Windows Server 2016 (10).
The WinCNG flakey tests should have a better chance when running on the
newer OS.
This update does not change the compiler mix.
Also change the test fixture to not use the `--quiet` option with the
`docker pull` commant. This option requires docker v19.03, and
AppVeyor's Visual Studio 2017 image doesn't support it. Log output did
not change without `--quiet`, so it seems safe to delete it. In case
we'd need it, another solution is to retry without `--quiet` if the
command fails. docker's exit status is 125 in that case.
Ref: https://github.com/libssh2/libssh2/issues/804#issuecomment-1515232799
Ref: https://www.appveyor.com/docs/windows-images-software/Closes#994
Keep a single list for mac and crypt algos that we use in both CMake
and autotools. Use the same test names across build tools.
Use the TAP protocol to track individual tests run from a single shell
script.
Also:
- enable the rest of our tests with autotools.
- set `make check` verbose to see errors in case they happen.
- silence stray 'command not found' error when running `mansyntax.sh`
on Windows.
GitHub Actions Windows docker tests disabled due to:
```
Command: docker build --quiet -t libssh2/openssh_server ../tests/openssh_server
Error running command 'docker build --quiet -t libssh2/openssh_server ../tests/openssh_server' (exit 1): Sending build context to Docker daemon 22.02kB
Step 1/42 : FROM debian:bullseye-slim
bullseye-slim: Pulling from library/debian
no matching manifest for windows/amd64 10.0.20348 in the manifest list entries
Failed to build docker image
```
Closes#993
Support for calculating MAC (message authentication code) on encrypted
data instead of plain text data.
This adds support for the following MACs:
- `hmac-sha1-etm@openssh.com`
- `hmac-sha2-256-etm@openssh.com`
- `hmac-sha2-512-etm@openssh.com`
Integration-patches-by: Viktor Szakats
* rebase on master
* fix checksec warnings
* fix compiler warning
* fix indent/whitespace/eol
* rebase/manual merge onto AES-GCM patch #797
* more manual merge of `libssh2_transport_send()` based
on dfandrich/shellfish
Fixes#582Closes#655Closes#987
Can send specific signals to remote process. Allows for slightly
improved remote process management, if the server supports it.
Integration-patches-by: Viktor Szakats
* doc updates
* change `signame_len` to `size_t`
* variable scopes
* fix checksrc warnings
Closes#672Closes#991
Add support for aes256-gcm@openssh.com and aes128-gcm@openssh.com
ciphers, which are the OpenSSH implementations of AES-GCM cryptography.
It is similar to RFC5647 but has changes to the MAC protocol
negotiation. These are implemented for recent versions of OpenSSL only.
The ciphers work differently than most previous ones in two big areas:
the cipher includes its own integrated MAC, and the packet length field
in the SSH frame is left unencrypted. The code changes necessary are
gated by flags in the LIBSSH2_CRYPT_METHOD configuration structure.
These differences mean that both the first and last parts of a block
require special handling during encryption. The first part is where the
packet length field is, which must be kept out of the encryption path
but in the authenticated part (as AAD). The last part is where the
Authentication Tag is found, which is calculated and appended during
encryption or removed and validated on decryption. As encryption/
decryption is performed on each packet in a loop, one block at a time,
flags indicating when the first and last blocks are being processed are
passed down to the encryption layers.
The strict block-by-block encryption that occurs with other protocols is
inappropriate for AES-GCM, since the packet length shifts the first
encrypted byte 4 bytes into the block. Additionally, the final part of
the block must contain the AES-GCM's Authentication Tag, so it must be
presented to the lower encryption layer whole. These requirements mean
added code to consolidate blocks as they are passed down.
When AES-GCM is negotiated as the cipher, its built-in MAC is
automatically used as the SSH MAC so further MAC negotiation is not
necessary. The SSH negotiation is skipped when _libssh2_mac_override()
indicates that such a cipher is in use. The virtual MAC configuration
block mac_method_hmac_aesgcm is then used as the MAC placeholder.
This work was sponsored by Anders Borum.
Integration-patches-by: Viktor Szakats
* fix checksrc errors
* fix openssl.c warning
* fix transport.c warnings
* switch to `LIBSSH2_MIN/MAX()` from `MIN()`/`MAX()`
* fix indent
* fix libgcrypt unused warning
* fix mbedtls unused warning
* fix wincng unused warning
* fix old openssl unused variable warnings
* delete blank lines
* updates to help merging with the ETM patch
Add new guard `LIBSSH2_RSA_SHA1`. Add missing guards for `LIBSSH2_RSA`,
`LIBSSH2_DSA`.
Fix warnings when all options are disabled.
This is still not complete and it's possible to break a build with
certain crypto backends (e.g. mbedTLS) and/or combination of options.
It's not guaranteed that all bits everywhere get disabled by these
settings. Consider this a "best effort".
Add these new options to disable certain crypto elements:
- `LIBSSH2_NO_3DES`
- `LIBSSH2_NO_AES_CTR`
- `LIBSSH2_NO_BLOWFISH`
- `LIBSSH2_NO_CAST`
- `LIBSSH2_NO_ECDSA`
- `LIBSSH2_NO_RC4`
- `LIBSSH2_NO_RSA_SHA1`
- `LIBSSH2_NO_RSA`
The goal is to offer a way to disable legacy/obsolete/insecure ones.
See also: 146a25a06d `LIBSSH2_NO_HMAC_RIPEMD`
See also: 38015f4e46 `LIBSSH2_NO_DSA`
See also: be31457f30 `LIBSSH2_NO_MD5`
Closes#986
This also significantly (by 7-8x in my limited tests with a short
string) speeds up this function. The impact is still minor as this
function is only used in `knownhost.c` in release builds.
Closes#985
Works like the `FIXTURE_TRACE_ALL` envvar, but enables full trace for
the connection phase only.
Also fix a possible NULL deref with `FIXTURE_TRACE_ALL` and a failed
`libssh2_session_init_ex()`.
Tested in #979
`CONFIGURATION` was never passed to the cmake command, so it had
never enabled logging when set to `Debug`.
Also `CONFIGURATION` is ambiguous depending on the "generator" used
by CMake. In case of Visual Studio, this is a build/ctest-time
setting, not a cmake-config parameter.
So set this permanently to `Release` and enable logging via our
dedicated CMake option `ENABLE_DEBUG_LOGGING`.
Tested in #979
- fix indentation errors.
- reformat `cmake/FindmbedTLS.cmake`
- replace a macro with a variable in `example/sftp_RW_nonblock.c`.
- delete macOS macro `_DARWIN_USE_64_BIT_INODE` from the
OS/400 config header, `os400/libssh2_config.h`.
- fix other minor nits.
Closes#983
- use `gman` alias if present. This makes it work when the correct `man`
command is provided via `brew` on macOS.
- move CMake attempts to detect tools necessary to run `mansyntax.sh`
into the script itself.
- delete CMake TODO to move more test logic into CMake. This would make
it CMake-specific and require maintaining it separately for each build
tool. Just use our external script when a POSIX shell is available.
Closes#982
Move `libssh2_config.h` generation and setting `-DHAVE_CONFIG_H` to
the root `CMakeFile.txt`.
Also move symbol hiding setup there. It needs to be done before
generating the config file for `LIBSSH2_API` value to be set in it.
After this change the `HIDE_SYMBOLS` setting is accepted without an
annoying CMake warning when not actually building a shared libssh2 lib.
Closes#981
Drop checks from Windows builds and enable it based on `WIN32`.
This saves detection time and also makes 3rd party builds simpler.
Also:
- delete `HAVE_DISABLED_NONBLOCKING`, that we used in build tools to
explicitly disable an explicit `#error` in `session.c`.
- replace existing `WSAEWOULDBLOCK` check for Windows support with
`WIN32`. Cleaner with the same result.
Follow-up to f1e80d8d8c
Follow-up to 5644eea216Closes#980
Make the public `libssh2_base64_decode()` a wrapper for that.
Bump up length sizes in callers.
Also fix output size calculation to first divide then multiply.
Closes#978
- rename tests to have more succint names and a more useful natural
order.
- rename `simple` and `ssh2` in tests to have the `test_` prefix.
This avoids a name collisions with `ssh2` in examples.
- cmake: drop the `example-` prefix for generated examples.
Bringing their names in sync with other build tools, like autotools.
- move common auth test code into the fixture and simplify tests by
using that.
- move feature guards from CMake to preprocessor for auth tests.
Now it works with all build tools and it's easier to keep it in sync
with the lib itself.
For this we need to include `libssh2_priv.h` in tests, which in turn
needs tweaking on the trick we use to suppress extra MSVS warnings
when building tests and examples.
- move mbedTLS blocklist for crypto tests from CMake to the test
fixture.
- add ed25519 hostkey tests to `test_hostkey` and `test_hostkey_hash`.
- add shell script to regenerate all test keys used for our tests.
- alpha-sort tests.
- rename `signed_*` keys to begin with `key` like the rest of the keys
do.
- whitespace fixes.
Closes#969
From debian:bullseye
- doesn't need manual bumps.
- is ahead of stable and should be stable enough for our purpose.
- slim is saving resources.
Closes#971
Skip testing unixy methods on Windows and vice versa.
I continue to assume that CMake doesn't define `WIN32` with Cygwin
(as Cygwin doesn't define `_WIN32`/`WIN32` for C), though I haven't
tested this.
Closes#970
A new flag named `LIBSSH2_FLAG_QUOTE_PATHS` has been added, to make
libssh2 not quote file paths sent to the remote's scp subsystem. Some
custom ssh daemons cannot handle quoted paths, and this makes this flag
useful.
Authored-by: Jörgen Sigvardsson <jorgen.sigvardsson@westermo.com>
