session: add libssh2_session_callback_set2()

Add new `libssh2_session_callback_set2()` API that deprecates
`libssh2_session_callback_set()`.

The new implementation offers the same functionality, but accepts and
returns a generic function pointer (of type `libssh2_cb_generic *`), as
opposed to the old function that used data pointers (`void *`). The new
solution thus avoids data to function (and vice versa) pointer
conversions, which has undefined behaviour in standard C.

About the name: It seems the `*2` suffix was used in the past for
replacement functions for deprecated ones. Let's stick with that.
`*_ex` was preferred for new functions that extend existing ones with
new features.

Closes #1285

docs/libssh2_session_callback_set2.3

@@ -0,0 +1,139 @@
+.\" Copyright (C) The libssh2 project and its contributors.
+.\" SPDX-License-Identifier: BSD-3-Clause
+.TH libssh2_session_callback_set2 3 "13 Dec 2023" "libssh2 1.11.1" "libssh2"
+.SH NAME
+libssh2_session_callback_set2 - set a callback function
+.SH SYNOPSIS
+.nf
+#include <libssh2.h>
+
+libssh2_cb_generic *
+libssh2_session_callback_set2(LIBSSH2_SESSION *session, int cbtype,
+                              libssh2_cb_generic *callback);
+.fi
+.SH DESCRIPTION
+Sets a custom callback handler for a previously initialized session
+object. Callbacks are triggered by the receipt of special packets at the
+Transport layer. To disable a callback, set it to NULL.
+
+\fIsession\fP - Session instance as returned by
+.BR libssh2_session_init_ex(3)
+
+\fIcbtype\fP - Callback type. One of the types listed in Callback Types.
+
+\fIcallback\fP - Pointer to custom callback function. The prototype for
+this function must match the associated callback declaration macro.
+.SH CALLBACK TYPES
+.IP LIBSSH2_CALLBACK_IGNORE
+Called when a SSH_MSG_IGNORE message is received
+.IP LIBSSH2_CALLBACK_DEBUG
+Called when a SSH_MSG_DEBUG message is received
+.IP LIBSSH2_CALLBACK_DISCONNECT
+Called when a SSH_MSG_DISCONNECT message is received
+.IP LIBSSH2_CALLBACK_MACERROR
+Called when a mismatched MAC has been detected in the transport layer. If the
+function returns 0, the packet will be accepted nonetheless.
+.IP LIBSSH2_CALLBACK_X11
+Called when an X11 connection has been accepted
+.IP LIBSSH2_CALLBACK_SEND
+Called when libssh2 wants to send data on the connection. Can be set to a
+custom function to handle I/O your own way.
+
+The prototype of the callback:
+
+.nf
+ssize_t sendcb(libssh2_socket_t sockfd, const void *buffer,
+               size_t length, int flags, void **abstract);
+.fi
+
+\fBsockfd\fP is the socket to write to, \fBbuffer\fP points to the data to
+send, \fBlength\fP is the size of the data, \fBflags\fP is the flags that
+would have been used to a \fIsend()\fP call and \fBabstract\fP is a pointer
+to the abstract pointer set in the \fIlibssh2_session_init_ex(3)\fP call.
+
+The callback returns the number of bytes sent, or \-1 for error. The special
+return code \fB-EAGAIN\fP can be returned to signal that the send was aborted
+to prevent getting blocked and it needs to be called again.
+.IP LIBSSH2_CALLBACK_RECV
+Called when libssh2 wants to read data from the connection. Can be set to a
+custom function to handle I/O your own way.
+
+The prototype of the callback:
+
+.nf
+ssize_t recvcb(libssh2_socket_t sockfd, void *buffer,
+               size_t length, int flags, void **abstract);
+.fi
+
+\fBsockfd\fP is the socket to read from, \fBbuffer\fP where to store received
+data into, \fBlength\fP is the size of the buffer, \fBflags\fP is the flags
+that would have been used to a \fIrecv()\fP call and \fBabstract\fP is a pointer
+to the abstract pointer set in the \fIlibssh2_session_init_ex(3)\fP call.
+
+The callback returns the number of bytes read, or \-1 for error. The special
+return code \fB-EAGAIN\fP can be returned to signal that the read was aborted
+to prevent getting blocked and it needs to be called again.
+.IP LIBSSH2_CALLBACK_AUTHAGENT
+Called during authentication process to allow the client to connect to the
+ssh-agent and perform any setup, such as configuring the agent or adding keys.
+
+The prototype of the callback:
+
+.nf
+void authagent(LIBSSH2_SESSION* session, LIBSSH2_CHANNEL *channel,
+               void **abstract);
+.fi
+.IP LIBSSH2_CALLBACK_AUTHAGENT_IDENTITIES
+Not called by libssh2. The client is responsible for calling this method when
+a SSH2_AGENTC_REQUEST_IDENTITIES message has been received.
+
+The prototype of the callback:
+
+.nf
+void identities(LIBSSH2_SESSION* session, void *buffer,
+                const char *agent_path,
+                void **abstract)
+.fi
+
+\fBbuffer\fP must be filled in by the callback. Different clients may implement
+this differently. For example, one client may pass in an unsigned char ** for
+this parameter, while another may pass in a pointer to a struct.
+
+Regardless of the type of buffer used, the client will need to send back a list
+of identities in the following format.
+
+uint32 buffer length
+uint32 number of entries
+entries
+
+Where each entry in the entries list is of the format:
+
+string data
+cstring comment
+
+\fBagent_path\fP The path to a running ssh-agent on the client machine, from
+which identities can be listed.
+.IP LIBSSH2_CALLBACK_AUTHAGENT_SIGN
+Not called by libssh2. The client is responsible for calling this method when
+a SSH2_AGENTC_SIGN_REQUEST message has been received.
+
+The prototype of the callback:
+
+.nf
+void sign(LIBSSH2_SESSION* session,
+          unsigned char *blob, unsigned int blen,
+          const unsigned char *data, unsigned int dlen,
+          unsigned char **sig, unsigned int *sig_len,
+          const char *agent_path,
+          void **abstract);
+.fi
+
+When interfacing with an ssh-agent installed on the client system, this method
+can call libssh2_agent_sign(3) to perform signing.
+
+.SH RETURN VALUE
+Pointer to previous callback handler. Returns NULL if no prior callback
+handler was set or the callback type was unknown.
+.SH SEE ALSO
+.BR libssh2_session_init_ex(3)
+.BR libssh2_agent_sign(3)