1
0
mirror of https://github.com/libssh2/libssh2.git synced 2025-07-05 20:21:11 +03:00

tests: improve running tests

TL;DR: Sync test builds between autotools and CMake. Sync sshd
configuration between Docker and non-Docker fixtures. Bump up
sshd_config for recent OpenSSH releases.

This also opens up the path to have non-Docker tests that use a
local sshd process. Though sshd is practically unusable on Windows
CI machines out of the box, so this will need further efforts.

Details:

- cmake: run sshd fixture test just like autotool did already.

- sync tests and their order between autotools and CMake.

  It makes `test_aa_warmup` the first test with both.

- cmake: load test lists from `Makefile.am`.

  Needed to update the loader to throw away certain lines to keep the
  converted output conform CMake syntax. Using regexp might be an
  alternative way of doing this, but couldn't make it work.

- cmake: use the official way to configure test environment variables.
  Switch to syntax that's extendable.

- cmake: allow to run the same test both under Docker and sshd fixture.

  Useful for testing the sshd fixture runner, or how the same test
  behaves in each fixture.

- update test fixture to read the username from `USER` envvar instead of
  using the Dockfile-specific hardwired one, when running outside Docker.

- rework `ssh2.sh` into `sshd_fixture.sh`, to:

  - allow running any tests (not just `test_ssh2`).
  - configure Docker tests for running outside Docker.
  - fixup `SSHD` path when running on Windows (e.g. in AppVeyor CI).
    Fixes: `sshd re-exec requires execution with an absolute path`
  - allow overriding `PUBKEY` and `PRIVKEY` envvars.
  - allow overriding `ssh_config` via `SSHD_FIXTURE_CONFIG`.

- prepare support for running multiple tests via sshd_fixture.

  Add a TAP runner for autotools and extend CMake logic. The TAP runner
  loads the test list from `Makefile.am`.

  Notice however that on Windows, `sshd_fixture.sh` is very flaky with
  GitHub Actions. And consistently broken for subsequent tests in
  AppVeyor CI:
    'libssh2_session_handshake failed (-43): Failed getting banner'

  Another way to try is a single sshd instance serving all tests.
  For CMake this would probably mean using an external script.

- ed25519 test keys were identical for auth and host. Regenerate the
  auth keypair to make them distinct.

- sync the sshd environment between Docker and sshd_fixture.

  - use common via `openssh_server/sshd_config`.
  - accept same auth keys.
  - offer the same host keys.
  - sync TrustedUserCAKeys.
  - delete now unused keypairs: `etc/host*`, `etc/user*`.
  - bump up startup delay for Windows (randomly, to 5 secs, from 3).
  - delete `UsePrivilegeSeparation no` to avoid deprecation warnings.
    `command-line line 0: Deprecated option UsePrivilegeSeparation`
  - delete `Protocol 2` to avoid deprecation warnings.
    It has been the default since OpenSSH 3.0 (2001-11-06).
  - delete `StrictModes no` (CI tests work without it, Docker tests
    never used it).

- bump `Dockerfile` base image to `testing-slim` (from `bullseye-slim`).

  It needed `sshd_config` updates to keep things working with
  OpenSSH 9.2 (compared to bullseye's 8.4).

  - replace `ChallengeResponseAuthentication` alias with
    `KbdInteractiveAuthentication`.
    The former is no longer present in default `sshd_config` since
    OpenSSH 8.7 (2021-08-20). This broke the `Dockerfile` script.
    The new name is documented since OpenSSH 4.9 (2008-03-31)

  - add `PubkeyAcceptedKeyTypes +ssh-rsa,ssh-dss,ssh-rsa-cert-v01@openssh.com`
    and `HostKeyAlgorithms +ssh-rsa`.

    Original-patch-by: Eric van Gyzen (@vangyzen on github)
    Fixes #691

    There is a new name for `PubkeyAcceptedKeyTypes`:
       `PubkeyAcceptedAlgorithms`.
    It requires OpenSSH 8.5 (2021-03-03) and breaks some envs so we're
    not using it just yet.

- drop `rijndael-cbc@lysator.liu.se` tests and references from config.

  This is a draft alias for `aes256-cbc`. No need to test it twice.
  Also this alias is no longer recognized by OpenSSH 8.5 (2021-03-03).

- update `mansyntax.sh` and `sshd_fixture.sh` to not rely on `srcdir`.

  Hopefully this works with out-of-tree builds.

- fix `test_read_algos.test` to honor CRLF EOLs in their inputs
  (necessary when running on Windows.)

- fix `test_read_algos.test` to honor `EXEEXT`. Might be useful when
  running tests under cross-builds?

- `test_ssh2.c`:

  - use libssh2 API to set blocking mode. This makes it support all
    platforms.
  - adapt socket open timeout logic from `openssh_fixture.c`.
    Sadly this did not help fix flakiness on GHA Windows.

- tests: delete unused C headers and variable initialization.

- delete unused test files: `sshd_fixture.sh.in`, `sshdwrap`,
  `etc/sshd_config`.

  Ref: cf80f2f4b5

- autotools: delete stray `.c` test sources from `EXTRA_DIST` in tests.

- `tests/.gitignore`: drop two stray tests.

- autotools: fix passing `SSHD` containing space (Windows needs this).

- autotools: sort `EXTRA_DIST` in tests.

- cmake: fix to add `test_ssh2` to `TEST_TARGETS`.

- fix `authorized_key` order in `tests/gen_keys.sh`.

- silence shellcheck warning in `ci/checksrc.sh`.

- set `SSHD` for autotools on GitHub Actions Windows. [skipped]

  Auto-detection doesn't work (maybe because sshd is installed via
  Git for Windows and we're using MSYS2's shell.)

  It enables running sshd fixture (non-Docker) tests in these jobs.

  I did not include this in the final patch due to flakiness:
  ```
  Connection to 127.0.0.1:4711 attempt #0 failed: retrying...
  Connection to 127.0.0.1:4711 attempt #1 failed: retrying...
  Connection to 127.0.0.1:4711 attempt #2 failed: retrying...
  Failure establishing SSH session: -43
  ```

  Can be enabled with:
  `export SSHD='C:/Program Files/Git/usr/bin/sshd.exe'`

Closes #996
This commit is contained in:
Viktor Szakats
2023-04-21 20:06:30 +00:00
parent fdf824d6f4
commit a459a25302
29 changed files with 288 additions and 322 deletions

View File

@ -248,7 +248,7 @@ int test_case(int num,
int rc;
char *message;
int error_code;
LIBSSH2_SESSION *session = NULL;
LIBSSH2_SESSION *session;
alloc_count = 0;
free_count = 0;