Fix device test environment variables (#6229)

* Fix device test environment variables

Device tests were not connecting properly to WiFi because the
environment variables were not set when WiFi.connect was called.
This would result in tests sometimes working *if* the prior sketch run
on the ESP saved WiFi connection information and auto-connect was
enabled.  But, in most cases, the tests would simply never connect to
any WiFi and fail.

getenv() works only after BS_RUN is called (because BS_RUN handles the
actual parsing of environment variables sent from the host).

Add a "pretest" function to all tests which is called by the host test
controller only after all environment variables are set.  Move all
WiFi/etc. operations that were in each separate test's setup() into it.

So the order of operations for tests now is:
ESP:  setup()
      -> Set serial baud
      -> Call BS_RUN()
HOST: Send environment
      Send "do pretest"
ESP:  pretest()
      -> Set Wifi using env. ariables, etc. return "true" on success
HOST: Send "run test 1"
ESP:  Run 1st test, return result
HOST: Send "run test 2"
ESP:  Run 2nd test, return result
<and so forth>

If nothing is needed to be set up, just return true from the pretest
function.

All tests now run and at least connect to WiFi.  There still seem to be
some actual test errors, but not because of the WiFi/environment
variables anymore.

* Remove unneeded debug prints

* Silence esptool.py output when not in V=1 mode

Esptool-ck.exe had an option to be silent, but esptool.py doesn't so the
output is very chatty and makes looking a the run logs hard (60 lines
of esptool.py output, 3 lines of actual test reports).

Redirect esptool.py STDOUT to /dev/null unless V=1 to clear this up.

* Speed up builds massively by removing old JSON

arduino-builder checks the build.options.json file and then goes off and
pegs my CPU at 100% for over a minute on each test compile checking if
files have been modified.

Simply deleting any pre-existing options.json file causes this step to
be skipped and a quick, clean recompile is done in siginificantly less
time.

* Enable compile warnings, fix any that show up

Enable all GCC warnings when building the tests and fix any that came up
(mostly signed/unsigned, unused, and deprecated ones).

* Fix UMM_MALLOC printf crash, umm_test

Printf can now handle PROGMEM addresses, so simplify and correct the
debug printouts in umm_info and elsewhere.

tests/device/libraries/BSTest/runner.py

@@ -104,6 +104,10 @@ class BSTestRunner(object):
                     if res != BSTestRunner.SUCCESS:
                         print('failed to set environment variables')
                         break;
+                    res = self.pretest()
+                    if res != BSTestRunner.SUCCESS:
+                        print('failed to run pretest init')
+                        break;
                     should_update_env = False
                 if name in self.mocks:
                     debug_print('setting up mocks')
@@ -198,6 +202,21 @@ class BSTestRunner(object):
                 return BSTestRunner.TIMEOUT
         return BSTestRunner.SUCCESS
 
+    def pretest(self):
+        # Environment now set up, call the pretest init (wifi connect, etc.)
+        self.sp.sendline('pretest');
+        timeout = 10
+        while timeout > 0:
+            res = self.sp.expect(['>>>>>bs_test_pretest result=1', EOF, TIMEOUT]) # Only expect a pass, abort testing if failure
+            if res == 0:
+                break
+            time.sleep(0.1)
+            timeout -= 0.1
+        if res != 0:
+            return BSTestRunner.TIMEOUT
+        else:
+            return BSTestRunner.SUCCESS
+
     def request_env(self, key):
         self.sp.sendline('getenv "{}"'.format(key))
         timeout = 10

tests/device/libraries/BSTest/src/BSArgs.h

@@ -69,7 +69,7 @@ inline size_t split_args(char *line, char **argv, size_t argv_size)
     const int ESCAPE = '\\';
     const int SPACE = ' ';
     split_state_t state = SS_SPACE;
-    int argc = 0;
+    size_t argc = 0;
     char *next_arg_start = line;
     char *out_ptr = line;
     for (char *in_ptr = line; argc < argv_size - 1; ++in_ptr) {
@@ -148,4 +148,4 @@ inline size_t split_args(char *line, char **argv, size_t argv_size)
 
 } // namespace protocol
 
-#endif //BS_ARGS_H
+#endif //BS_ARGS_H

tests/device/libraries/BSTest/src/BSProtocol.h

@@ -5,6 +5,8 @@
 
 #define BS_LINE_PREFIX ">>>>>bs_test_"
 
+extern bool pretest();
+
 namespace bs
 {
 namespace protocol
@@ -58,6 +60,12 @@ void output_getenv_result(IO& io, const char* key, const char* value)
     io.printf(BS_LINE_PREFIX "getenv value=\"%s\"\n", value);
 }
 
+template<typename IO>
+void output_pretest_result(IO& io, bool res)
+{
+    io.printf(BS_LINE_PREFIX "pretest result=%d\n", res?1:0);
+}
+
 template<typename IO>
 bool input_handle(IO& io, char* line_buf, size_t line_buf_size, int& test_num)
 {
@@ -87,6 +95,14 @@ bool input_handle(IO& io, char* line_buf, size_t line_buf_size, int& test_num)
         output_getenv_result(io, argv[1], (value != NULL) ? value : "");
         return false;
     }
+    if (strcmp(argv[0], "pretest") == 0) {
+        if (argc != 1) {
+            return false;
+        }
+        bool res = ::pretest();
+        output_pretest_result(io, res);
+        return false;
+    }
     /* not one of the commands, try to parse as test number */
     char* endptr;
     test_num = (int) strtol(argv[0], &endptr, 10);