Make OS/400 implementation work again (#953)

* os400: support QADRT development files in a non-standard directory

This enables the possibility to compile libssh2 even if the ascii
runtime development files are not installed system-wide.

* userauth_kbd_packet: fix a pointer target type mismatch.

A temporary variable matching the parameter type is used before copying
to the real target and checking for overflow (that should not occur!).

* os400qc3: move and fix big number procedures

A bug added by a previous code style cleaning is fixed.
_libssh2_random() now checks and return the success status.

* os400qc3: fix cipher definition block lengths

They were wrongly set to the key size.

* Diffie-Hellman min/max modulus sizes are dependent of crypto-backend

In particular, os400qc3 limits the maximum group size to 2048-bits.
Move definitions of these parameters to crypto backend header files.

* kex: return an error if Diffie-Hellman key pair generation fails

* os400: add an ascii assert.h header file

* os400qc3: implement RSA SHA2 256/512

src/kex.c

@@ -263,8 +263,11 @@ static int diffie_hellman_sha_algo(LIBSSH2_SESSION *session,
 
         rc = libssh2_dh_key_pair(&exchange_state->x, exchange_state->e, g, p,
                                  group_order, exchange_state->ctx);
-        if(rc)
+        if(rc) {
+            ret = _libssh2_error(session, LIBSSH2_ERROR_KEX_FAILURE,
+                                 "dh key pair generation failed");
             goto clean_exit;
+        }
 
         /* Send KEX init */
         /* packet_type(1) + String Length(4) + leading 0(1) */