This PR contains a series of patches that date back many years and I
believe were discussed on the mailing list, but never merged. We have
been using these in our local copy of libssh2 without issue since 2015,
if not earlier. I believe this is the full set of changes, as we tried
to use comments to mark where our copy of libssh2 differs from the
canonical version.
This also contains changes I made earlier this year, but which were not
discussed on the mailing list, to support certificates and FIDO2 keys
with agent forwarding.
Note that this is not a complete implementation of agent forwarding, as
that is outside the scope of libssh2. Clients still need to provide
their own implementation that parses ssh-agent methods after calling
libssh2_channel_read() and calls the appropriate callback messages in
libssh2. See the man page changes in this PR for more details.
Integration-patches-by: Viktor Szakats
* prefer size_t
* prefer unsigned int over u_int in public function
* add const
* docs, indent, checksrc, debug call, compiler warning fixes
Implement picky warnings with clang in autotools. Extend picky gcc
warnings, sync them between build tools and compilers and greatly
speed up detection in CMake.
- autotools: enable clang compiler warnings with `--enable-debug`.
- autotools: enable more gcc compiler warnings with `--enable-debug`.
- autotools/cmake: sync compiler warning options between gcc and clang.
- sync compiler warning options between autotools and cmake.
- cmake: reduce option-checks to speed up the detection phase.
Bring them down to 3 (from 35). Leaving some checks to keep the
CMake logic alive and for an easy way to add new options.
clang 3.0 (2011-11-29) and gcc 2.95 (1999-07-31) now required.
- autotools logic copied from curl, with these differences:
- delete `-Wimplicit-fallthrough=4` due to a false positive.
- reduce `-Wformat-truncation=2` to `1` due to a false positive.
- simplify MinGW detection for `-Wno-pedantic-ms-format`.
- cmake: show enabled picky compiler options (like autotools).
- cmake: do compile `tests/simple.c` and `tests/ssh2.c`.
- fix new compiler warnings.
- `tests/CMakeLists.txt`: fix indentation.
Original source of autotools logic:
- a8fbdb461c/acinclude.m4
- a8fbdb461c/m4/curl-compilers.m4
Notice that the autotools implementation considers Apple clang as
legacy clang 3.7. CMake detection works more accurately, at the same
time more error-prone and difficult to update due to the sparsely
documented nature of Apple clang option evolution.
Closes#952
- introduce the concept of a project level setup header
`src/libssh2_setup.h`, that is used by `src`, `example` and `tests`
alike. Move there all common platform/compiler configuration from
`src/libssh2_priv.h`, individual sources and `CMakeFiles.txt` files.
Also move there our hand-crafted (= not auto-generated by CMake or
autotools) configuration `win32/libssh2-config.h`.
- `win32` directory is empty now, delete it.
- `Makefile.mk`: adapt to the above. Build-directory is the target
triplet, or any custom name set via `BLD_DIR`.
- sync header path order between build systems:
build/src -> source/src -> source/include
- delete redundant references to `windows.h`, `winsock2.h`,
`ws2tcpip.h`.
- delete unnecessary #includes, update order (`libssh2_setup.h` first,
`winsock2.h` first), simplify where possible.
This makes the code warning-free without `WIN32_LEAN_AND_MEAN`.
At the same time this patch applies this macro globally, to avoid
header bloat.
- example: add missing *nix header guards.
- example: fix misindented `HAVE_UNISTD_H` `#ifdef`s.
- set `WIN32` with all build-tools.
- set `HAVE_SYS_PARAM_H` in the hand-crafted config for MinGW.
To match auto-detection.
- move a source-specific macro to `misc.c` from `libssh2_priv.h`.
See the PR's individual commits for step-by-step updates.
Closes#932
Most of the changes aim to silence warnings by adding casts.
An assortment of other issues, mainly compiler warnings, resolved:
- unreachable code fixed by using `goto` in
`publickey_response_success()` in `publickey.c`.
- potentially uninitialized variable in `sftp_open()`.
- MSVS-specific bogus warnings with `nid_type` in `kex.c`.
- check result of `kex_session_ecdh_curve_type()`.
- add missing function declarations.
- type changes to fit values without casts:
- `cmd_len` in `scp_recv()` and `scp_send()`: `int` -> `size_t`
- `Blowfish_expandstate()`, `Blowfish_expand0state()` loop counters:
`uint16_t` -> `int`
- `RECV_SEND_ALL()`: `int` -> `ssize_t`
- `shell_quotearg()` -> `unsigned` -> `size_t`
- `sig_len` in `_libssh2_mbedtls_rsa_sha2_sign()`:
`unsigned` -> `size_t`
- `prefs_len` in `libssh2_session_method_pref()`: `int` -> `size_t`
- `firstsec` in `_libssh2_debug_low()`: `int` -> `long`
- `method_len` in `libssh2_session_method_pref()`: `int` -> `size_t`
- simplify `_libssh2_ntohu64()`.
- fix `LIBSSH2_INT64_T_FORMAT` for MinGW.
- fix gcc warning by not using a bit field for
`burn_optimistic_kexinit`.
- fix unused variable warning in `_libssh2_cipher_crypt()` in
`libgcrypt.c`.
- fix unused variables with `HAVE_DISABLED_NONBLOCKING`.
- avoid const stripping with `BIO_new_mem_buf()` and OpenSSL 1.0.2 and
newer.
- add a missing const in `wincng.h`.
- FIXME added for public:
- `libssh2_channel_window_read_ex()` `read_avail` argument type.
- `libssh2_base64_decode()` `datalen` argument type.
- fix possible overflow in `sftp_read()`.
Ref: 4552c73cd5
- formatting in `wincng.h`.
See warning details in the PR's individual commits.
Cherry-picked from #846Closes#876
Before this patch, with debug logging disabled, libssh2 code used a
variadic macro to catch `_libssh2_debug()` calls, and convert them to
no-ops. In certain conditions, it used an empty inline function instead.
Variadic macro is a C99 feature. It means that depending on compiler,
and build settings, it littered the build log with warnings about this.
The new solution uses the trick of passing the variable arg list as a
single argument and pass that down to the debug function with a regular
macro. When disabled, another regular C89-compatible macro converts it
to a no-op.
This makes inlining, C99 variadic macros and maintaining the conditions
for each unnecessary and also makes the codebase compile more
consistently, e.g. with forced C standards and/or picky warnings.
TL;DR: It makes this feature C89-compliant.
- cmake: always link `ws2_32` on Windows. Also add it to `libssh2.pc`.
Fixes#745
- agent: fix gcc compiler warning:
`src/agent.c:296:35: warning: 'snprintf' output truncated before the last format character [-Wformat-truncation=]`
- autotools: fix `EVP_aes_128_ctr` detection with binutils `ld`
The prerequisite for a successful detection is setting
`LIBS=-lbcrypt` if the chosen openssl-compatible library requires
it, e.g. libressl, or quictls/openssl built with
`-DUSE_BCRYPTGENRANDOM`.
With llvm `lld`, detection works out of the box. With binutils `ld`,
it does not. The reason is `ld`s world-famous pickiness with lib
order.
To fix it, we pass all custom libs before and after the TLS libs.
This ugly hack makes `ld` happy and detection succeed.
- agent: fix Windows-specific warning:
`src/agent.c:318:10: warning: implicit conversion loses integer precision: 'LRESULT' (aka 'long long') to 'int' [-Wshorten-64-to-32]`
- src: fix llvm/clang compiler warning:
`src/libssh2_priv.h:987:28: warning: variadic macros are a C99 feature [-Wvariadic-macros]`
- src: support `inline` with `__GNUC__` (llvm/clang and gcc), fixing:
```
src/libssh2_priv.h:990:8: warning: extension used [-Wlanguage-extension-token]
static inline void
^
```
- blowfish: support `inline` keyword with MSVC.
Also switch to `__inline__` (from `__inline`) for `__GNUC__`:
https://gcc.gnu.org/onlinedocs/gcc/Inline.htmlhttps://clang.llvm.org/docs/UsersManual.html#differences-between-various-standard-modes
- example/test: fix MSVC compiler warnings:
- `example\direct_tcpip.c(209): warning C4244: 'function': conversion from 'unsigned int' to 'u_short', possible loss of data`
- `tests\session_fixture.c(96): warning C4013: 'getcwd' undefined; assuming extern returning int`
- `tests\session_fixture.c(100): warning C4013: 'chdir' undefined; assuming extern returning int`
- delete unused macros:
- `HAVE_SOCKET`
- `HAVE_INET_ADDR`
- `NEED_LIB_NSL`
- `NEED_LIB_SOCKET`
- `HAVE_NTSTATUS_H`
- `HAVE_NTDEF_H`
- build: delete stale zlib/openssl version numbers from path defaults.
- cmake: convert tabs to spaces, add newline at EOFs.
Closes#811
- convert `_libssh2_explicit_zero()` to macro. This allows inlining
where supported (e.g. `SecureZeroMemory()`).
- replace `SecureZeroMemory()` (in `wincng.c`) and
`LIBSSH2_CLEAR_MEMORY`-guarded `memset()` (in `os400qc3.c`) with
`_libssh2_explicit_zero()` macro.
- delete `LIBSSH2_CLEAR_MEMORY` guards, which enables secure-zeroing
universally.
- add `LIBSSH2_NO_CLEAR_MEMORY` option to disable secure-zeroing.
- while here, delete double/triple inclusion of `misc.h`.
`libssh2_priv.h` included it already.
Closes#810
- in `hostkey.c` check the result of `libssh2_sha256_init()` and
`libssh2_sha512_init()` calls. This avoid the warning that we're
ignoring the return values.
- fix code using `int` (or `SOCKET`) for sockets. Use libssh2's
dedicated `libssh2_socket_t` and `LIBSSH2_INVALID_SOCKET` instead.
- fix compiler warnings due to `STATUS_*` macro redefinitions between
`ntstatus.h` / `winnt.h`. Solve it by manually defining the single
`STATUS` value we need from `ntstatus.h` and stop including the whole
header.
Fixes#733
- improve Windows UWP/WinRT builds by detecting it with code copied
from the curl project. Then excluding problematic libssh2 parts
according to PR by Dmitry Kostjučenko.
Fixes#734
- always use `SecureZeroMemory()` on Windows.
We can tweak this if not found or not inlined by a C compiler which
we otherwise support. Same if it causes issues with UWP apps.
Ref: https://learn.microsoft.com/en-us/previous-versions/windows/desktop/legacy/aa366877(v=vs.85)
Ref: https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-rtlsecurezeromemory
- always enable `LIBSSH2_CLEAR_MEMORY` on Windows. CMake and
curl-for-win builds already did that. Delete `SecureZeroMemory()`
detection from autotools' WinCNG backend logic, that this
setting used to depend on.
TODO: Enable it for all platforms in a separate PR.
TODO: For clearing buffers in WinCNG, call `_libssh2_explicit_zero()`,
insead of a local function or explicit `SecureZeroMemory()`.
- Makefile.inc: move `os400qc3.h` to `HEADERS`. This fixes
compilation on non-unixy platforms. Recent regression.
- `libssh2.rc`: replace copyright with plain ASCII, as in curl.
Ref: curl/curl@1ca62bb
Ref: curl/curl#7765
Ref: curl/curl#7776
- CMake fixes and improvements:
- enable warnings with llvm/clang.
- enable more comprehensive warnings with gcc and llvm/clang.
Logic copied from curl:
233810bb5f/CMakeLists.txt (L131-L148)
- fix `Policy CMP0080` CMake warning by deleting that reference.
- add `ENABLE_WERROR` (default: `OFF`) option. Ported from curl.
- add `PICKY_COMPILER` (default: `ON`) option, as known from curl.
It controls both the newly added picky warnings for llvm/clang and
gcc, and also the pre-existing ones for MSVC.
- `win32/GNUmakefile` fixes and improvements:
- delete `_AMD64_` and add missing `-m64` for x64 builds under test.
- add support for `ARCH=custom`.
It disables hardcoded Intel 64-bit and Intel 32-bit options,
allowing ARM64 builds.
- add support for `LIBSSH2_RCFLAG_EXTRAS`.
To pass custom options to windres, e.g. in ARM64 builds.
- add support for `LIBSSH2_RC`. To override `windres`.
- delete support for Metrowerks C. Last released in 2004.
- `win32/libssh2_config.h`: delete unnecessary socket #includes
`src/libssh2_priv.h` includes `winsock2.h` and `ws2tcpip.h` further
down the line, triggered by `HAVE_WINSOCK2_H`.
`mswsock.h` does not seem to be necessary anymore.
Double-including these (before `windows.h`) caused compiler failures
when building against BoringSSL and warnings with LibreSSL. We could
work this around by passing `-DNOCRYPT`. Deleting the duplicates
fixes these issues.
Timeline:
2013: c910cd382d deleted `mswsock.h` from `src/libssh2_priv.h`
2008: 8c43bc52b1 added `winsock2.h` and `ws2tcpip.h` to `src/libssh2_priv.h`
2005: dc4bb1af96 added the now deleted #includes
- delete or replace `LIBSSH2_WIN32` with `WIN32`.
- replace hand-rolled `HAVE_WINDOWS_H` macro with `WIN32`. Also delete
its detections/definitions.
- delete unused `LIBSSH2_DARWIN` macro.
- delete unused `writev()` Windows implementation
There is no reference to `writev()` since 2007-02-02, commit
9d55db6501.
- fix a bunch of MSVC / llvm/clang / gcc compiler warnings:
- `warning C4100: '...': unreferenced formal parameter`
- using value of undefined PP macro `LIBSSH2DEBUG`
- missing void from function definition
- `if()` block missing in non-debug builds
- unreferenced variable in non-debug builds
- `warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]`
in `_libssh2_debug()`
- `warning C4295: 'ciphertext' : array is too small to include a terminating null character`
- `warning C4706: assignment within conditional expression`
- `warning C4996: 'inet_addr': Use inet_pton() or InetPton() instead or
define _WINSOCK_DEPRECATED_NO_WARNINGS to disable deprecated API warnings`
By suppressning it. Would be best to use inet_pton() as suggested.
On Windows this needs Vista though.
- `warning C4152: nonstandard extension, function/data pointer conversion in expression`
(silenced locally)
- `warning C4068: unknown pragma`
Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46354480/job/j7d0m34qgq8rag5wCloses#808
Files: libssh2.h, agent.c, userauth.c
Notes:
Part 2 of the fix for #659. This adds rsa key downgrading for agents that don't support sha2 upgrading. It also adds better trace output for debugging/logging around key upgrading.
Credit:
Will Cosgrove (signed off by Michael Buckley)
Files: agent.c, agent.h, agent_win.c
Notes:
* agent: Add support for Windows OpenSSH agent
The implementation was partially taken and modified from that found in
the Portable OpenSSH port to Win32 by the PowerShell team, but mostly
based on the existing Unix OpenSSH agent support.
https://github.com/PowerShell/openssh-portable
Regarding the partial transfer support implementation: partial transfers
are easy to deal with, but you need to track additional state when
non-blocking IO enters the picture. A tracker of how many bytes have
been transfered has been placed in the transfer context struct as that's
where it makes most sense. This tracker isn't placed behind a WIN32
#ifdef as it will probably be useful for other agent implementations.
* agent: win32 openssh: Disable overlapped IO
Non-blocking IO is not currently supported by the surrounding agent
code, despite a lot of the code having everything set up to handle it.
Credit:
Co-authored-by: Gabriel Smith <gabriel.smith@precisionot.com>
Files : agent.c
Notes :
Currently the error details as returned by agent_transact_pageant() are overwritten by a generic "agent list id failed" message by int agent_list_identities(LIBSSH2_AGENT* agent).
Credit :
Zenju
File : agent.c
Notes :
Libssh2 uses the SSH_AUTH_SOCK env variable to read the system agent location. However, when using a custom agent path you have to set this value using setenv which is not thread-safe. The new functions allow for a way to set a custom agent socket path in a thread safe manor.
Use checksrc.pl from the curl project, with (for now)
suppressed long line warnings and indentation set to
4 spaces. Fixes are whitespace for the most part.
Warning count went down from 2704 to 12.
Also fix codespell typos, two non-ANSI C89 comments
and a stray tab in include/libssh2.h.
Ref: https://github.com/libssh2/libssh2/pull/235
"agent_disconnect_unix", called by "libssh2_agent_disconnect", was
leaving the file descriptor in the agent structure unchanged. Later,
"libssh2_agent_free" would call again "libssh2_agent_disconnect" under
the hood and it would try to close again the same file descriptor. In
most cases that resulted in just a harmless error, but it is also
possible that the file descriptor had been reused between the two
calls resulting in the closing of an unrelated file descriptor.
This patch sets agent->fd to LIBSSH2_INVALID_SOCKET avoiding that
issue.
Signed-off-by: Salvador Fandiño <sfandino@yahoo.com>
Some structure fields holding callback addresses have the same name as the
underlying system function (connect, send, recv). Set parentheses around
their reference to suppress a possible macro substitution.
Use a macro for connect() on OS/400 to resolve a const/nonconst parameter
problem.
Fixes VS2012 code analysis warning C6387: 'p+4' may be '0':
this does not adhere to the specification for the function
'memcpy': libssh2 agent.c 330
Fixes VS2012 code analysis warning C6387: 'p' may be '0':
this does not adhere to the specification for the function
'UnmapViewOfFile': libssh2 agent.c 333
INVALID_SOCKET is a special value in Windows representing a
non-valid socket identifier. We were #defining this to -1 on
non-Windows platforms, causing unneccessary namespace pollution.
Let's have our own identifier instead.
Thanks to Matt Lawson for pointing this out.
Starting now, we unconditionally use the internal replacement functions
for send() and recv() - creatively named _libssh2_recv() and
_libssh2_send().
On errors, these functions return the negative 'errno' value instead of
the traditional -1. This design allows systems that have no "natural"
errno support to not have to invent it. It also means that no code
outside of these two transfer functions should use the errno variable.
As pointed out in bug report #173, this module basically never
used _libssh2_error() which made it work inconstently with other
parts of the libssh2 code base. This is my first take at making
this code more in line with the rest.
agent->ops gets initialized by the libssh2_agent_connect() call
but we need to make sure that we don't segfault even if a bad
sequence of function calls is used.
As pointed out by Grubsky Grigory <g.grubsky@securitycode.ru>, I
made a mistake when I added the _libssh2_store_str() call before
and I made a slightly different patch than what he suggested.
Based purely on taste.
To get the blocking vs non-blocking to work as smooth as possible
and behave better internally, we avoid using the external
interfaces when calling functions internally.
Renamed a few internal functions to use _libssh2 prefix when not
being private within a file, and removed the libssh2_ for one
that was private within the file.
I'll introduce a new internal function set named
_libssh2_store_u32
_libssh2_store_u64
_libssh2_store_str
That can be used all through the library to build binary outgoing
packets. Using these instead of the current approach removes
hundreds of lines from the library while at the same time greatly
enhances readability. I've not yet fully converted everything to
use these functions.
I've converted LOTS of 'unsigned long' to 'size_t' where
data/string lengths are dealt with internally. This is The Right
Thing and it will help us make the transition to our
size_t-polished API later on as well.
I'm removing the PACKET_* error codes. They were originally
introduced as a set of separate error codes from the transport
layer, but having its own set of errors turned out to be very
awkward and they were then converted into a set of #defines that
simply maps them to the global libssh2 error codes instead. Now,
I'l take the next logical step and simply replace the PACKET_*
defines with the actual LIBSSH2_ERROR_* defines. It will increase
readability and decrease confusion.
I also separated packet stuff into its own packet.h header file.
We reserve ^libssh2_ for public symbols and we use _libssh2 as
prefix for internal ones. I fixed the intendation of all these
edits with emacs afterwards, which then changed it slightly more
than just _libssh2_error() expressions but I didn't see any
obvious problems.
libssh2_error() no longer allocates a string and only accepts a const
error string. I also made a lot of functions use the construct of
return libssh2_error(...) instead of having one call to
libssh2_error() and then a separate return call. In several of those
cases I then also changed the former -1 return code to a more
detailed one - something that I think will not change behaviors
anywhere but it's worth keeping an eye open for any such.
