Add BearSSL client and server, support true bidir, lower memory, modern SSL (#4273)

BearSSL (https://www.bearssl.org) is a TLS(SSL) library written by
Thomas Pornin that is optimized for lower-memory embedded systems
like the ESP8266. It supports a wide variety of modern ciphers and
is unique in that it doesn't perform any memory allocations during
operation (which is the unfortunate bane of the current axTLS).

BearSSL is also absolutely focused on security and by default performs
all its security checks on x.509 certificates during the connection
phase (but if you want to be insecure and dangerous, that's possible
too).

While it does support unidirectional SSL buffers, like axTLS,
as implemented the ESP8266 wrappers only support bidirectional
buffers. These bidirectional buffers avoid deadlocks in protocols
which don't have well separated receive and transmit periods.

This patch adds several classes which allow connecting to TLS servers
using this library in almost the same way as axTLS:
BearSSL::WiFiClientSecure - WiFiClient that supports TLS
BearSSL::WiFiServerSecure - WiFiServer supporting TLS and client certs

It also introduces objects for PEM/DER encoded keys and certificates:
BearSSLX509List - x.509 Certificate (list) for general use
BearSSLPrivateKey - RSA or EC private key
BearSSLPublicKey - RSA or EC public key (i.e. from a public website)

Finally, it adds a Certificate Authority store object which lets
BearSSL access a set of trusted CA certificates on SPIFFS to allow it
to verify the identity of any remote site on the Internet, without
requiring RAM except for the single matching certificate.
CertStoreSPIFFSBearSSL - Certificate store utility

Client certificates are supported for the BearSSL::WiFiClientSecure, and
what's more the BearSSL::WiFiServerSecure can also *require* remote clients
to have a trusted certificate signed by a specific CA (or yourself with
self-signing CAs).

Maximum Fragment Length Negotiation probing and usage are supported, but
be aware that most sites on the Internet don't support it yet.  When
available, you can reduce the memory footprint of the SSL client or server
dramatically (i.e. down to 2-8KB vs. the ~22KB required for a full 16K
receive fragment and 512b send fragment).  You can also manually set a
smaller fragment size and guarantee at your protocol level all data will
fit within it.

Examples are included to show the usage of these new features.

axTLS has been moved to its own namespace, "axtls".  A default "using"
clause allows existing apps to run using axTLS without any changes.

The BearSSL::WiFi{client,server}Secure implements the axTLS
client/server API which lets many end user applications take advantage
of BearSSL with few or no changes.

The BearSSL static library used presently is stored at
https://github.com/earlephilhower/bearssl-esp8266 and can be built
using the standard ESP8266 toolchain.

libraries/ESP8266WiFi/examples/BearSSL_CertStore/BearSSL_CertStore.ino

@@ -0,0 +1,158 @@
+// Demonstrate the CertStore object with WiFiClientBearSSL
+//
+// Before running, you must download the set of certs using
+// the script "certs-from-mozilla.py" (no parameters)
+// and then uploading the generated data directory to
+// SPIFFS.
+//
+// Why would you need a CertStore?
+//
+// If you know the exact serve being connected to, or you
+// are generating your own self-signed certificates and aren't
+// allowing connections to HTTPS/TLS servers out of your
+// control, then you do NOT want a CertStore.  Hardcode the
+// self-signing CA or the site's x.509 certificate directly.
+//
+// However, if you don't know what specific sites the system
+// will be required to connect to and verify, a
+// CertStore{SPIFFS,SD}BearSSL can allow you to select from
+// 10s or 100s of CAs against which you can check the
+// target's X.509, without taking any more RAM than a single
+// certificate.  This is the same way that standard browsers
+// and operating systems use to verify SSL connections.
+//
+// About the chosen certs:
+// The certificates are scraped from the Mozilla.org current
+// list, but please don't take this as an endorsement or a
+// requirement:  it is up to YOU, the USER, to specify the
+// certificate authorities you will use as trust bases.
+//
+// Mar 2018 by Earle F. Philhower, III
+// Released to the public domain
+
+#include <ESP8266WiFi.h>
+#include <CertStoreSPIFFSBearSSL.h>
+#include <time.h>
+
+const char *ssid = "....";
+const char *pass = "....";
+
+// A single, global CertStore which can be used by all
+// connections.  Needs to stay live the entire time any of
+// the WiFiClientBearSSLs are present.
+CertStoreSPIFFSBearSSL certStore;
+
+// Set time via NTP, as required for x.509 validation
+void setClock() {
+  configTime(3 * 3600, 0, "pool.ntp.org", "time.nist.gov");
+
+  Serial.print("Waiting for NTP time sync: ");
+  time_t now = time(nullptr);
+  while (now < 8 * 3600 * 2) {
+    delay(500);
+    Serial.print(".");
+    now = time(nullptr);
+  }
+  Serial.println("");
+  struct tm timeinfo;
+  gmtime_r(&now, &timeinfo);
+  Serial.print("Current time: ");
+  Serial.print(asctime(&timeinfo));
+}
+
+// Try and connect using a WiFiClientBearSSL to specified host:port and dump URL
+void fetchURL(BearSSL::WiFiClientSecure *client, const char *host, const uint16_t port, const char *path) {
+  if (!path) {
+    path = "/";
+  }
+
+  Serial.printf("Trying: %s:443...", host);
+  client->connect(host, port);
+  if (!client->connected()) {
+    Serial.printf("*** Can't connect. ***\n-------\n");
+    return;
+  }
+  Serial.printf("Connected!\n-------\n");
+  client->write("GET ");
+  client->write(path);
+  client->write(" HTTP/1.0\r\nHost: ");
+  client->write(host);
+  client->write("\r\nUser-Agent: ESP8266\r\n");
+  client->write("\r\n");
+  uint32_t to = millis() + 5000;
+  if (client->connected()) {
+    do {
+      char tmp[32];
+      memset(tmp, 0, 32);
+      int rlen = client->read((uint8_t*)tmp, sizeof(tmp) - 1);
+      yield();
+      if (rlen < 0) {
+        break;
+      }
+      // Only print out first line up to \r, then abort connection
+      char *nl = strchr(tmp, '\r');
+      if (nl) {
+        *nl = 0;
+        Serial.print(tmp);
+        break;
+      }
+      Serial.print(tmp);
+    } while (millis() < to);
+  }
+  client->stop();
+  Serial.printf("\n-------\n");
+}
+
+void setup() {
+  Serial.begin(115200);
+  Serial.println();
+  Serial.println();
+
+  // We start by connecting to a WiFi network
+  Serial.print("Connecting to ");
+  Serial.println(ssid);
+  WiFi.mode(WIFI_STA);
+  WiFi.begin(ssid, pass);
+
+  while (WiFi.status() != WL_CONNECTED) {
+    delay(500);
+    Serial.print(".");
+  }
+  Serial.println("");
+
+  Serial.println("WiFi connected");
+  Serial.println("IP address: ");
+  Serial.println(WiFi.localIP());
+
+  setClock(); // Required for X.509 validation
+
+  int numCerts = certStore.initCertStore();
+  Serial.printf("Number of CA certs read: %d\n", numCerts);
+  if (numCerts == 0) {
+    Serial.printf("No certs found. Did you run certs-from-mozill.py and upload the SPIFFS directory before running?\n");
+    return; // Can't connect to anything w/o certs!
+  }
+
+  BearSSL::WiFiClientSecure *bear = new BearSSL::WiFiClientSecure();
+  // Integrate the cert store with this connection
+  bear->setCertStore(&certStore);
+  Serial.printf("Attempting to fetch https://www.github.com/...\n");
+  fetchURL(bear, "www.github.com", 443, "/");
+  delete bear;
+}
+
+void loop() {
+  Serial.printf("\nPlease enter a website address (www.blah.com) to connect to: ");
+  String site;
+  do {
+    site = Serial.readString();
+  } while (site == "");
+  Serial.printf("https://%s/\n", site.c_str());
+
+  BearSSL::WiFiClientSecure *bear = new BearSSL::WiFiClientSecure();
+  // Integrate the cert store with this connection
+  bear->setCertStore(&certStore);
+  fetchURL(bear, site.c_str(), 443, "/");
+  delete bear;
+}
+

libraries/ESP8266WiFi/examples/BearSSL_CertStore/certs-from-mozilla.py

@@ -0,0 +1,51 @@
+#!/usr/bin/python
+
+# This script pulls the list of Mozilla trusted certificate authorities
+# from the web at the "mozurl" below, parses the file to grab the PEM
+# for each cert, and then generates DER files in a new ./data directory
+# Upload these to a SPIFFS filesystem and use the CertManager to parse
+# and use them for your outgoing SSL connections.
+#
+# Script by Earle F. Philhower, III.  Released to the public domain.
+
+import csv
+from os import mkdir
+from subprocess import Popen, PIPE
+import urllib2
+try:
+    # for Python 2.x
+    from StringIO import StringIO
+except ImportError:
+    # for Python 3.x
+    from io import StringIO
+
+# Mozilla's URL for the CSV file with included PEM certs
+mozurl = "https://ccadb-public.secure.force.com/mozilla/IncludedCACertificateReportPEMCSV"
+
+# Load the manes[] and pems[] array from the URL
+names = []
+pems = []
+response = urllib2.urlopen(mozurl)
+csvData = response.read()
+csvReader = csv.reader(StringIO(csvData))
+for row in csvReader:
+    names.append(row[0]+":"+row[1]+":"+row[2])
+    pems.append(row[28])
+del names[0] # Remove headers
+del pems[0] # Remove headers
+
+# Try and make ./data, skip if present
+try:
+    os.mkdir("data")
+except:
+    pass
+
+# Process the text PEM using openssl into DER files
+for i in range(0, len(pems)):
+    certName = "data/ca_%03d.der" % (i);
+    thisPem = pems[i].replace("'", "")
+    print names[i] + " -> " + certName
+    pipe = Popen(['openssl','x509','-inform','PEM','-outform','DER','-out', certName], shell = False, stdin = PIPE).stdin
+    pipe.write(thisPem)
+    pipe.close
+