To reflect the support for multiple container engines, and to make
terminology more consistent and implementation-agnostic.
Also:
- add FIXME for `docker-machine ip` command that has been dropped
upstream.
Follow-up to 9bd1f16dc2#2605Closes#2609
To render in Git webviews as-is, to make it easier to edit, verify,
and to sync up with curl.
- add options to not build the `.3` man pages:
- autotools: `--disable-docs`
- cmake: `LIBSSH2_BUILD_DOCS=OFF`
- building `.3` man pages requires Perl after this patch.
- drop `mansyntax` and the shell / `grep` / GNU `man` tool requirements with it.
- scripts and most logic were copied from curl.
- add `cd2nroff` from curl, with edits to relax curl-specific checks.
- used `nroff2cd` (from curl) to convert from `.3` to `.md`. Then
manually fixed copyrights, inline function references and a couple
of other things.
Credits-to: Daniel Stenberg
Ref: eefcc1bda4
Ref: https://github.com/curl/curl/pull/12730
Ref: https://github.com/libssh2/www/issues/25#issuecomment-3289431671Closes#1660
Use `AM_CFLAGS` to pass custom, per-target C flags. This replaces using
`CFLAGS` which triggered this warning when running `autoreconf -fi`:
```
tests/Makefile.am:8: warning: 'CFLAGS' is a user variable, you should not override it;
tests/Makefile.am:8: use 'AM_CFLAGS' instead
```
(Only for `tests`, even though `example` and `src` also used this
method. The warning is also missing from curl, that also uses
`CFLAGS`.)
Follow-up to 3ec53f3ea2#1286Closes#1378
- All files have prominent copyright and SPDX identifier
- If not embedded in the file, in the .reuse/dep5 file
- All used licenses are in LICENSES/ (not shipped in tarballs)
- A new REUSE CI job verify that all files are OK
Assisted-by: Viktor Szakats
Closes#1084
`DYN=1` means to build examples/tests against the shared libssh2.
Before this patch this was broken for building tests. This patch skips
building tests that require the static libssh2 library, so the build now
succeeds.
Also move the list of tests that require static lib from
`CMakeLists.txt` to `Makefile.inc`, so that we can reuse it in
`Makefile.mk`.
Couldn't find a way to also reuse it in `Makefile.am`. Move the
`Makefile.am` specific definitions close to the shared list, to make it
easier to keep them synced.
Cherry-picked from #1036
Before this patch libssh2 used a variety of solutions to pass the source
directory to tests: `FIXTURE_WORKDIR` build-time macro (cmake),
`FIXTURE_WORKDIR` envvar (unused), setting `srcdir` manually
(autotools), setting current directory (cmake), and also `builddir`
envvar (autotools) for passing current working dir to `mansyntax.sh`.
This patch reduces this to using existing `srcdir` with autotools and
setting it ourselves in CMake. This was mostly enabled by this recent
patch: 4c9ed51f96
Details:
- cmake: replace baked-in `FIXTURE_WORKDIR` macro with env.
Added in 54bef4c5da#198 (2018-03-21)
- rename `FIXTURE_WORKDIR` to `srcdir`, to match autotools.
- cmake: add missing `srcdir` for algo and sshd tests.
- session_fixture: stop `chdir()`-ing, rely on prefixing with `srcdir`.
Changing current directory should be unnecessary after
4c9ed51f96#801 (2023-02-24),
that prefixes referenced input filenames with the `srcdir` envvar.
The `srcdir` envvar was already exported by autotools, and now we're
also setting it from CMake.
- cmake: stop setting `WORKING_DIRECTORY`, rely on `srcdir` env.
`WORKING_DIRECTORY` is no longer necessary, after passing `srcdir` to
all tests, so they can find our source tree and keys/etc in it
regardless of the current directory.
Also this past commit hints that `WORKING_DIRECTORY` wasn't always
working for this purpose as expected:
"tests: Xcode doesn't obey CMake's test working directory"
Ref: 10a5cbf945
- autotools: delete explicit `srcdir` for test env.
Added in 13f8addd1b (2015-07-02)
automake documents `srcdir` as exported to the test environment:
c04c4e8856/doc/automake.texi (L9302-L9304)https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html
It's mentioned in the docs back in 1997 and got a regression test in
2012. We can safely assume it to be available without setting it
ourselves.
- autotools: delete explicit `builddir`.
Added in 13f8addd1b (2015-07-02)
It seems this wasn't necessary to make the above fix work, and
`mansyntax.sh` is able to figure out the build workdir by reading
`$PWD`. Our out-of-tree and `make distcheck` CI builds also work
without it.
Let us know if there is a scenario we're missing and needs this.
Closes#1032
autotools:
- `--disable-docker-tests`
- `--disable-sshd-tests`
cmake:
- `RUN_DOCKER_TESTS`
- `RUN_SSHD_TESTS`
Update automake and ci to use this new flag and delete former logic
of relying on Windows detection and `HOST_WINDOWS`. Also fix honoring
this when running `test_read_algos.test`.
This allows to disable these individually and on per-CI/local-job basis.
To run as much tests as the env allows.
Cherry-picked from #1017
Instead of assembling the list using `$(wildcard ...)`.
Also split off a `tests/Makefile.inc` from `tests/Makefile.am`. With its
simpler syntax, this also allows to delete some complexity from the
CMake loader.
Cherry-picked from #1017
- add #included C files to `EXTRA_DIST`.
Regression from 4f0f4bff5a
- fix `tests/sshd_fixture.sh` to not write into the test dir, by using
a pre-assembled `TrustedUserCAKeys` file. Update `Dockerfile` too to
use this.
Regression from a459a25302
Also update `tests/sshd_fixture.sh` to use
`openssh_server/authorized_keys` like `Dockerfile` does. And a few more
cosmetic updates.
Closes#1006
TL;DR: Sync test builds between autotools and CMake. Sync sshd
configuration between Docker and non-Docker fixtures. Bump up
sshd_config for recent OpenSSH releases.
This also opens up the path to have non-Docker tests that use a
local sshd process. Though sshd is practically unusable on Windows
CI machines out of the box, so this will need further efforts.
Details:
- cmake: run sshd fixture test just like autotool did already.
- sync tests and their order between autotools and CMake.
It makes `test_aa_warmup` the first test with both.
- cmake: load test lists from `Makefile.am`.
Needed to update the loader to throw away certain lines to keep the
converted output conform CMake syntax. Using regexp might be an
alternative way of doing this, but couldn't make it work.
- cmake: use the official way to configure test environment variables.
Switch to syntax that's extendable.
- cmake: allow to run the same test both under Docker and sshd fixture.
Useful for testing the sshd fixture runner, or how the same test
behaves in each fixture.
- update test fixture to read the username from `USER` envvar instead of
using the Dockfile-specific hardwired one, when running outside Docker.
- rework `ssh2.sh` into `sshd_fixture.sh`, to:
- allow running any tests (not just `test_ssh2`).
- configure Docker tests for running outside Docker.
- fixup `SSHD` path when running on Windows (e.g. in AppVeyor CI).
Fixes: `sshd re-exec requires execution with an absolute path`
- allow overriding `PUBKEY` and `PRIVKEY` envvars.
- allow overriding `ssh_config` via `SSHD_FIXTURE_CONFIG`.
- prepare support for running multiple tests via sshd_fixture.
Add a TAP runner for autotools and extend CMake logic. The TAP runner
loads the test list from `Makefile.am`.
Notice however that on Windows, `sshd_fixture.sh` is very flaky with
GitHub Actions. And consistently broken for subsequent tests in
AppVeyor CI:
'libssh2_session_handshake failed (-43): Failed getting banner'
Another way to try is a single sshd instance serving all tests.
For CMake this would probably mean using an external script.
- ed25519 test keys were identical for auth and host. Regenerate the
auth keypair to make them distinct.
- sync the sshd environment between Docker and sshd_fixture.
- use common via `openssh_server/sshd_config`.
- accept same auth keys.
- offer the same host keys.
- sync TrustedUserCAKeys.
- delete now unused keypairs: `etc/host*`, `etc/user*`.
- bump up startup delay for Windows (randomly, to 5 secs, from 3).
- delete `UsePrivilegeSeparation no` to avoid deprecation warnings.
`command-line line 0: Deprecated option UsePrivilegeSeparation`
- delete `Protocol 2` to avoid deprecation warnings.
It has been the default since OpenSSH 3.0 (2001-11-06).
- delete `StrictModes no` (CI tests work without it, Docker tests
never used it).
- bump `Dockerfile` base image to `testing-slim` (from `bullseye-slim`).
It needed `sshd_config` updates to keep things working with
OpenSSH 9.2 (compared to bullseye's 8.4).
- replace `ChallengeResponseAuthentication` alias with
`KbdInteractiveAuthentication`.
The former is no longer present in default `sshd_config` since
OpenSSH 8.7 (2021-08-20). This broke the `Dockerfile` script.
The new name is documented since OpenSSH 4.9 (2008-03-31)
- add `PubkeyAcceptedKeyTypes +ssh-rsa,ssh-dss,ssh-rsa-cert-v01@openssh.com`
and `HostKeyAlgorithms +ssh-rsa`.
Original-patch-by: Eric van Gyzen (@vangyzen on github)
Fixes#691
There is a new name for `PubkeyAcceptedKeyTypes`:
`PubkeyAcceptedAlgorithms`.
It requires OpenSSH 8.5 (2021-03-03) and breaks some envs so we're
not using it just yet.
- drop `rijndael-cbc@lysator.liu.se` tests and references from config.
This is a draft alias for `aes256-cbc`. No need to test it twice.
Also this alias is no longer recognized by OpenSSH 8.5 (2021-03-03).
- update `mansyntax.sh` and `sshd_fixture.sh` to not rely on `srcdir`.
Hopefully this works with out-of-tree builds.
- fix `test_read_algos.test` to honor CRLF EOLs in their inputs
(necessary when running on Windows.)
- fix `test_read_algos.test` to honor `EXEEXT`. Might be useful when
running tests under cross-builds?
- `test_ssh2.c`:
- use libssh2 API to set blocking mode. This makes it support all
platforms.
- adapt socket open timeout logic from `openssh_fixture.c`.
Sadly this did not help fix flakiness on GHA Windows.
- tests: delete unused C headers and variable initialization.
- delete unused test files: `sshd_fixture.sh.in`, `sshdwrap`,
`etc/sshd_config`.
Ref: cf80f2f4b5
- autotools: delete stray `.c` test sources from `EXTRA_DIST` in tests.
- `tests/.gitignore`: drop two stray tests.
- autotools: fix passing `SSHD` containing space (Windows needs this).
- autotools: sort `EXTRA_DIST` in tests.
- cmake: fix to add `test_ssh2` to `TEST_TARGETS`.
- fix `authorized_key` order in `tests/gen_keys.sh`.
- silence shellcheck warning in `ci/checksrc.sh`.
- set `SSHD` for autotools on GitHub Actions Windows. [skipped]
Auto-detection doesn't work (maybe because sshd is installed via
Git for Windows and we're using MSYS2's shell.)
It enables running sshd fixture (non-Docker) tests in these jobs.
I did not include this in the final patch due to flakiness:
```
Connection to 127.0.0.1:4711 attempt #0 failed: retrying...
Connection to 127.0.0.1:4711 attempt #1 failed: retrying...
Connection to 127.0.0.1:4711 attempt #2 failed: retrying...
Failure establishing SSH session: -43
```
Can be enabled with:
`export SSHD='C:/Program Files/Git/usr/bin/sshd.exe'`
Closes#996
Keep a single list for mac and crypt algos that we use in both CMake
and autotools. Use the same test names across build tools.
Use the TAP protocol to track individual tests run from a single shell
script.
Also:
- enable the rest of our tests with autotools.
- set `make check` verbose to see errors in case they happen.
- silence stray 'command not found' error when running `mansyntax.sh`
on Windows.
GitHub Actions Windows docker tests disabled due to:
```
Command: docker build --quiet -t libssh2/openssh_server ../tests/openssh_server
Error running command 'docker build --quiet -t libssh2/openssh_server ../tests/openssh_server' (exit 1): Sending build context to Docker daemon 22.02kB
Step 1/42 : FROM debian:bullseye-slim
bullseye-slim: Pulling from library/debian
no matching manifest for windows/amd64 10.0.20348 in the manifest list entries
Failed to build docker image
```
Closes#993
Support for calculating MAC (message authentication code) on encrypted
data instead of plain text data.
This adds support for the following MACs:
- `hmac-sha1-etm@openssh.com`
- `hmac-sha2-256-etm@openssh.com`
- `hmac-sha2-512-etm@openssh.com`
Integration-patches-by: Viktor Szakats
* rebase on master
* fix checksec warnings
* fix compiler warning
* fix indent/whitespace/eol
* rebase/manual merge onto AES-GCM patch #797
* more manual merge of `libssh2_transport_send()` based
on dfandrich/shellfish
Fixes#582Closes#655Closes#987
Make the public `libssh2_base64_decode()` a wrapper for that.
Bump up length sizes in callers.
Also fix output size calculation to first divide then multiply.
Closes#978
- rename tests to have more succint names and a more useful natural
order.
- rename `simple` and `ssh2` in tests to have the `test_` prefix.
This avoids a name collisions with `ssh2` in examples.
- cmake: drop the `example-` prefix for generated examples.
Bringing their names in sync with other build tools, like autotools.
- move common auth test code into the fixture and simplify tests by
using that.
- move feature guards from CMake to preprocessor for auth tests.
Now it works with all build tools and it's easier to keep it in sync
with the lib itself.
For this we need to include `libssh2_priv.h` in tests, which in turn
needs tweaking on the trick we use to suppress extra MSVS warnings
when building tests and examples.
- move mbedTLS blocklist for crypto tests from CMake to the test
fixture.
- add ed25519 hostkey tests to `test_hostkey` and `test_hostkey_hash`.
- add shell script to regenerate all test keys used for our tests.
- alpha-sort tests.
- rename `signed_*` keys to begin with `key` like the rest of the keys
do.
- whitespace fixes.
Closes#969
- introduce the concept of a project level setup header
`src/libssh2_setup.h`, that is used by `src`, `example` and `tests`
alike. Move there all common platform/compiler configuration from
`src/libssh2_priv.h`, individual sources and `CMakeFiles.txt` files.
Also move there our hand-crafted (= not auto-generated by CMake or
autotools) configuration `win32/libssh2-config.h`.
- `win32` directory is empty now, delete it.
- `Makefile.mk`: adapt to the above. Build-directory is the target
triplet, or any custom name set via `BLD_DIR`.
- sync header path order between build systems:
build/src -> source/src -> source/include
- delete redundant references to `windows.h`, `winsock2.h`,
`ws2tcpip.h`.
- delete unnecessary #includes, update order (`libssh2_setup.h` first,
`winsock2.h` first), simplify where possible.
This makes the code warning-free without `WIN32_LEAN_AND_MEAN`.
At the same time this patch applies this macro globally, to avoid
header bloat.
- example: add missing *nix header guards.
- example: fix misindented `HAVE_UNISTD_H` `#ifdef`s.
- set `WIN32` with all build-tools.
- set `HAVE_SYS_PARAM_H` in the hand-crafted config for MinGW.
To match auto-detection.
- move a source-specific macro to `misc.c` from `libssh2_priv.h`.
See the PR's individual commits for step-by-step updates.
Closes#932
Before this patch CMake did feature detections in three files:
`src/CMakefiles.txt`, `examples/CMakefiles.txt` and
`tests/CMakefiles.txt`.
Merge and move them to the root `CMakefiles.txt`.
After this patch we end up with a single `src/libssh2_config.h`. This
brings CMake in sync with autotools builds, which already worked with
a single config header.
This also prevents mistakes where feature detection went out of sync
between `src` & `tests` (see ae90a35d15).
`tests` do compile sources from `src` directly, so these should always
be in sync.
It also allows to better integrate hand-crafted, platform-specific
config headers into the builds, like the one currently residing in
the `win32` directory (and also in `vms` and `os400`). Subject to an
upcoming PR.
Also fix a warning revealed after this patch made CMake correctly
enable `HAVE_GETTIMEOFDAY` for `example` programs.
Closes#906
Flakiness got continously worse these last days. It didn't seem related
to recent commits. Flakiness also picked up in GitHub CI runs, something
rarely seen before. Manual restart consistently fixed them.
The repeating pattern was the _first_ test (`test_hostkey`) failing,
with `libssh2_session_handshake failed (-13): Failed getting banner`.
Failures came after a lengthy wait, suggesting a timeout.
I then reversed the order of the first two tests, and it turned out that
the _first_ test failed again (`test_hostkey_hash`). Also pointing to a
timeout issue.
Then I added a dummy test to "warm up" whatever needs warming up in the
layers of CI + Docker + ssh server and their interconnects. This helped,
and GitHub CI tests run without failure right for the first time.
AppVeyor CI also improved a little.
This patch adds a new first test called `test_warmup`, that creates a
new libssh2 session, and exits with success even if that attempt failed.
A stop-gap solution at best, and there is no guarantee it will continue
to fix this or similar future issues, but it's also untenable to have
almost every CI run fail for intermittent reasons.
In some [1] cases [2] it's not the first test failing intermittently.
That's a different issue, and this patch doesn't fix it.
[1] #804
[2] https://ci.appveyor.com/project/libssh2org/libssh2/builds/46440828/job/8rej6cq6itg7vc4w#L500
Improve tests added in 7487dcf4b4 by
running `test_read` commands directly. This makes external shell/batch
files unnecessary, and is friendlier with cross-builds and when run
from non-default shells, like MSYS2.
Also extend CRYPT/MAC test error messages with the CRYPT/MAC name.
External runner shell scripts kept for future use.
Closes#814
Connects to the ssh server then downloads several MB of data. This
tests the data transfer path as well as boundary cases in packet
handling as data is split into smaller SSH blocks.
Many files have been added to the cmake build files but not the automake
ones in recent years. Missing ones have been added so automake "make
dist" will now create a usable tar ball.
The integration tests using Docker are now built with automake as well
(with "make check"). They are not run yet since they aren't working yet
on Linux.
Files: hostkey.c, userauth.c, test_public_key_auth_succeeds_with_correct_ecdsa_key.c
Notes:
Support ECDSA certificate authentication
Add a test for:
- Existing ecdsa basic public key authentication
- ecdsa public key authentication with a signed public key
Credit:
kkoenig
File:
Makefile.am
Notes:
No longer conditionally include OpenSSL specific test files, they aren't run if we're not building against OpenSSL 1.1.x anyway.
Credit:
Daniel Stenberg
Files:
.travis.yml, configure.ac, ossfuzz
Notes:
This adds support for an OSS-Fuzz fuzzing target in ssh2_client_fuzzer,
which is a cut down example of ssh2.c. Future enhancements can improve
coverage.
Credit:
Max Dymond
OpenSSH Key and ED25519 support #39
Added _libssh2_explicit_zero() to explicitly zero sensitive data in memory #120
* ED25519 Key file support - Requires OpenSSL 1.1.1 or later
* OpenSSH Key format reading support - Supports RSA/DSA/ECDSA/ED25519 types
* New string buffer reading functions - These add build-in bounds checking and convenance methods. Used for OpenSSL PEM file reading.
* Added new tests for OpenSSH formatted Keys
This introduces a test suite for libssh2. It runs OpenSSH in a Docker
container because that works well on Windows (via docker-machine) as
well as Linux. Presumably it works on Mac too with docker-machine, but
I've not tested that.
Because the test suite is docker-machine aware, you can also run it
against a cloud provider, for more realistic network testing, by setting
your cloud provider as your active docker machine. The Appveyor CI setup
in this commit does that because Appveyor doesn't support docker
locally.
Despite we announced the CMake support in libssh2-1.6.0 release notes,
the files required by the CMake build system were not included in the
release tarballs. Hence, the only way to use CMake for build was the
upstream git repository.
This commit makes CMake actually supported in the release tarballs.
Do not create symbolic links off the build directory. Recent autotools
verify that out-of-source build works even if the source directory tree
is not writable.
1 - let libssh2 compile with OpenSSL 0.9.6b. This is due to 'crypt' is found
in one of its header files and EVP_MAX_BLOCK_LENGTH not being found.
2 - The EXEEXT patch is because automake 1.7 doesn't support it, and recent
automakes add it automatically
