1
0
mirror of https://github.com/esp8266/Arduino.git synced 2025-04-22 21:23:07 +03:00

181 Commits

Author SHA1 Message Date
Earle F. Philhower, III
09896d5287 Add String::concat(char*, len) to allow non null-term strings (#6754)
* Add comcat(char*, len) to Sting

Fixes #5061

Adds a concat(const char *data, int len) method which allows arbitrary
sequences of data (including ones w/embedded \0s) to be appended to a
String.  May be useful for certain MQTT operations.

Adds sanity test for the feature to host suite

* Review comment cleanups
2019-11-14 00:58:07 +01:00
Develo
453eb2d064
Add wait loop at the end of mode, refactor can_yield (#6721)
* Add wait loop at the end of mode, refactor can_yield

* fix mock build
2019-11-07 18:25:20 -03:00
Earle F. Philhower, III
5d0b9aa5ac
Remove typo in device test "make clean" (#6673)
VIM strikes again.
2019-10-26 16:45:24 -07:00
Earle F. Philhower, III
41ba21613d
Fix Python3 errors for device tests (#6670)
* Fix Python3 errors for device tests

The Python3 migration didn't include fixes for local scripts in the
device test tree.

Fatal build and run Python errors fixed.

The last update to xunitmerge is ~5 years ago, so it looks to be
unsupported now.

Use a local copy of the two components to allow patching to work with
Python3.

The serial test seems to send garbage chars (non-ASCII/etc.), so use a
codepage 437 which supports all 255 chars.

Fixes: #6660

* Run tests at 160MHz (req'd for some SSL connections)

* Fix debuglevel options for builder

* Fix Python3 interpreter path in xunitmerge

* Remove virtualenv on "make clean"

* Add appropriate attribution, license to xunitmerge

Add like to the original sources with the author's license to the
copied/fixed xunitmerge files.
2019-10-26 13:58:54 -07:00
Earle F. Philhower, III
40f456aca3
Dynamically find the Python3 dir on Windows (#6646)
Windows installers place each version of Python in their own, uniquely
named directory.  Can't programatically override w/free Chocolatey, so
use the shell and figure it where it installed it dynamically.
2019-10-15 09:21:18 -07:00
Earle F. Philhower, III
2a5d215977
Reduce the IRAM usage of I2C code by 600-1500 bytes (#6326)
* Reduce the IRAM (and heap) usage of I2C code

The I2C code takes a large chunk of IRAM space.  Attempt to reduce the
size of the routines without impacting functionality.

First, remove the `static` classifier on the sda/scl variables in the
event handlers.  The first instructions in the routines overwrite the
last value stored in them, anyway, and their addresses are never taken.

* Make most variables ints, not uint8_ts

Where it doesn't make a functional difference, make global variables
ints and not unit8_t.  Bytewide updates and extracts require multiple
instructions and hence increase IRAM usage as well as runtime.

* Make local flag vars int

Sketch uses 270855 bytes (25%) of program storage space. Maximum is 1044464 bytes.
Global variables use 27940 bytes (34%) of dynamic memory, leaving 53980 bytes for local variables. Maximum is 81920 bytes.
./xtensa-lx106-elf/bin/xtensa-lx106-elf-objdump -t  -j .text1 /tmp/arduino_build_9615/*elf | sort -k1 | head -20
401000cc l     F .text1	00000014 twi_delay
401000ec l     F .text1	00000020 twi_reply$part$1
4010010c g     F .text1	00000035 twi_reply
4010014c g     F .text1	00000052 twi_stop
401001a0 g     F .text1	0000003b twi_releaseBus
40100204 g     F .text1	000001e6 twi_onTwipEvent
40100404 l     F .text1	000001f7 onSdaChange
40100608 l     F .text1	000002fd onSclChange
40100908 l     F .text1	0000003b onTimer

* Factor out !scl in onSdaChange

If SCL is low then all branches of the case are no-ops, so factor that
portion outo to remove some redundant logic in each case.

Sketch uses 270843 bytes (25%) of program storage space. Maximum is 1044464 bytes.
Global variables use 27944 bytes (34%) of dynamic memory, leaving 53976 bytes for local variables. Maximum is 81920 bytes.

401000cc l     F .text1	00000014 twi_delay
401000ec l     F .text1	00000020 twi_reply$part$1
4010010c g     F .text1	00000035 twi_reply
4010014c g     F .text1	00000052 twi_stop
401001a0 g     F .text1	0000003b twi_releaseBus
40100204 g     F .text1	000001e6 twi_onTwipEvent
40100404 l     F .text1	000001e7 onSdaChange
401005f8 l     F .text1	000002fd onSclChange
401008f8 l     F .text1	0000003b onTimer

0x0000000040107468                _text_end = ABSOLUTE (.)

* Make tiny twi_reply inline

twi_reply is a chunk of code that can be inlined and actually save IRAM
space because certain conditions acan be statically evaluated by gcc.

Sketch uses 270823 bytes (25%) of program storage space. Maximum is 1044464 bytes.
Global variables use 27944 bytes (34%) of dynamic memory, leaving 53976 bytes for local variables. Maximum is 81920 bytes.

401000cc l     F .text1	00000014 twi_delay
401000f4 g     F .text1	00000052 twi_stop
40100148 g     F .text1	0000003b twi_releaseBus
401001b0 g     F .text1	00000206 twi_onTwipEvent
401003d0 l     F .text1	000001e7 onSdaChange
401005c4 l     F .text1	000002fd onSclChange
401008c4 l     F .text1	0000003b onTimer
40100918 g     F .text1	00000085 millis
401009a0 g     F .text1	0000000f micros
401009b0 g     F .text1	00000022 micros64
401009d8 g     F .text1	00000013 delayMicroseconds
401009f0 g     F .text1	00000034 __digitalRead
401009f0  w    F .text1	00000034 digitalRead
40100a3c g     F .text1	000000e4 interrupt_handler
40100b20 g     F .text1	0000000f vPortFree

0x0000000040107434                _text_end = ABSOLUTE (.)

* Inline additional twi_** helper functions

Sketch uses 270799 bytes (25%) of program storage space. Maximum is 1044464 bytes.
Global variables use 27944 bytes (34%) of dynamic memory, leaving 53976 bytes for local variables. Maximum is 81920 bytes.

401000cc l     F .text1	00000014 twi_delay
401000f4  w    F .text1	0000003b twi_releaseBus
4010015c g     F .text1	00000246 twi_onTwipEvent
401003bc l     F .text1	000001e7 onSdaChange
401005b0 l     F .text1	000002f9 onSclChange
401008ac l     F .text1	0000003b onTimer

0x000000004010741c                _text_end = ABSOLUTE (.)

* Convert state machine to 1-hot for faster lookup

GCC won't use a lookup table for the TWI state machine, so it ends up
using a series of straight line compare-jump, compare-jumps to figure
out which branch of code to execute for each state.  For branches that
have multiple states that call them, this can expand to a lot of code.

Short-circuit the whole thing by converting the FSM to a 1-hot encoding
while executing it, and then just and-ing the 1-hot state with the
bitmask of states with the same code.

Sketch uses 270719 bytes (25%) of program storage space. Maximum is 1044464 bytes.
Global variables use 27944 bytes (34%) of dynamic memory, leaving 53976 bytes for local variables. Maximum is 81920 bytes.

401000cc l     F .text1	00000014 twi_delay
401000f4  w    F .text1	0000003b twi_releaseBus
4010015c g     F .text1	00000246 twi_onTwipEvent
401003c0 l     F .text1	000001b1 onSdaChange
40100580 l     F .text1	000002da onSclChange
4010085c l     F .text1	0000003b onTimer

0x00000000401073cc                _text_end = ABSOLUTE (.)

Saves 228 bytes of IRAM vs. master, uses 32 additional bytes of heap.

* Factor out twi_status setting

twi_status is set immediately before  an event handler is called,
resulting in lots of duplicated code.  Set the twi_status flag inside
the handler itself.

Saves an add'l ~100 bytes of IRAM from prior changes, for a total of
~340 bytes.

earle@server:~/Arduino/hardware/esp8266com/esp8266/tools$ ./xtensa-lx106-elf/bin/xtensa-lx106-elf-objdump -t  -j .text1 /tmp/arduino_build_849115/*elf | sort -k1 | head -20

401000cc l     F .text1	00000014 twi_delay
401000f4  w    F .text1	0000003b twi_releaseBus
40100160 g     F .text1	0000024e twi_onTwipEvent
401003c8 l     F .text1	00000181 onSdaChange
40100558 l     F .text1	00000297 onSclChange

* Use a struct to hold globals for TWI

Thanks to the suggestion from @mhightower83, move all global objects
into a struct.  This lets a single base pointer register to be used in
place of constantly reloading the address of each individual variable.

This might be better expressed by moving this to a real C++
implementaion based on a class object (the twi.xxxx would go back to the
old xxx-only naming for vars), but there would then need to be API
wrappers since the functionality is exposed through a plain C API.

Saves 168 additional code bytes, for a grand total of 550 bytes IRAM.

earle@server:~/Arduino/hardware/esp8266com/esp8266/tools$ ./xtensa-lx106-elf/bin/xtensa-lx106-elf-objdump -t  -j .text1 /tmp/arduino_build_849115/*elf | sort -k1 | head -20

401000cc l     F .text1	00000014 twi_delay
401000e8  w    F .text1	00000032 twi_releaseBus
40100128 g     F .text1	00000217 twi_onTwipEvent
4010034c l     F .text1	00000149 onSdaChange
4010049c l     F .text1	00000267 onSclChange
40100704 l     F .text1	00000028 onTimer

* Use enums for states, move one more var to twi struct

Make the TWI states enums and not #defines, in the hope that it will
allow GCC to more easily flag problems and general good code
organization.

401000cc l     F .text1	00000014 twi_delay
401000e8  w    F .text1	00000032 twi_releaseBus
40100128 g     F .text1	00000217 twi_onTwipEvent
4010034c l     F .text1	00000149 onSdaChange
4010049c l     F .text1	00000257 onSclChange
401006f4 l     F .text1	00000028 onTimer

Looks like another 16 bytes IRAM saved from the prior push.

Sketch uses 267079 bytes (25%) of program storage space. Maximum is 1044464 bytes.
Global variables use 27696 bytes (33%) of dynamic memory, leaving 54224 bytes for local variables. Maximum is 81920 bytes.

* Save 4 heap bytes by reprdering struct

* Convert to C++ class, clean up code

Convert the entire file into a C++ class (with C wrappers to preserve
the ABI).  This allows for setting individual values of the global
struct(class) in-situ instead of a cryptic list at the end of the struct
definition.  It also removes a lot of redundant `twi.`s from most class
members.

Clean up the code by converting from `#defines` to inline functions, get
rid of ternarys-as-ifs, use real enums, etc.

For slave_receiver.ino, the numbers are:
GIT Master IRAM: 0x723c
This push IRAM: 0x6fc0

For a savings of 636 total IRAM bytes (note, there may be a slight flash
text increase, but we have 1MB of flash to work with and only 32K of IRAM
so the tradeoff makes sense.

* Run astyle core.conf, clean up space/tab/etc.

Since the C++ version has significant text differences anyway, now is a
good time to clean up the mess of spaces, tabs, and differing cuddles.

* Add enum use comment, rename twi::delay, fix SDA/SCL_READ bool usage

Per review comments

* Replace clock stretch repeated code w/inline loop

There were multiple places where the code was waiting for a slave to
finish stretching the clock.  Factor them out to an *inline* function
to reduce code smell.

* Remove slave code when not using slave mode

Add a new twi_setSlaveMode call which actually attached the interrupts
to the slave pin change code onSdaChenge/onSclChange.  Don't attach
interrupts in the main twi_begin.

Because slave mode is only useful should a onoReceive or onRequest
callback, call twi_setSlaveMode and attach interrupts on the Wire
setters.

This allows GCC to not link in slave code unless slave mode is used,
saving over 1,000 bytes of IRAM in the common, master-only case.
2019-10-14 14:32:41 -07:00
Allman-astyler
9bdcd4f36a Allman astyle: mDNS (#6629)
* allmanize mDNS
2019-10-10 23:30:06 +02:00
david gauchard
8e0aac92c0 CI: trusty (u14.04) -> bionic (u18.04) (#6627)
* no-op test

* no-op test

* bionic

* missing python package for doc building with bionic

* fix previous commit

* remove external repo for gcc7 (causing issues), it is default under bionic
fix python3-setuptools install

* tryfix git submodule recursive (git 2.21/2.22 ?)
2019-10-10 12:30:59 -03:00
dsv19
8c3f1be63f Serial.flush modification (#5293)
* Modify Serial.flush

* Add function to uart tests
2019-10-08 05:36:39 -07:00
david gauchard
d4757542e4
use arduino IDE stable in CI (#6572)
* fix CI by using a stable arduino/builder version
* get rid of --allow-unauthenticated issue
2019-10-01 00:42:46 +02:00
Earle F. Philhower, III
8dd068eb40
Add memmove_P, use it in String to ensure F() safety (#6514)
memmove_P is now in libc, so use it to allow WString to handle F()
pointers without errors.

Supercedes #6368

Fixes #6384
2019-09-13 15:33:16 -07:00
david gauchard
0474eb9943
emulation on host: show timestamp on console output (#6507)
* emulation on host: option to add timestamp on console output
2019-09-12 14:00:27 +02:00
Earle F. Philhower, III
1f86311d79
Add Win32 build to CI system (#6493)
Build a single INO under a Windows VM on Travis to ensure
that the Win32 toolchain works properly.

In build.py, instead of making a string with spaces and then
splitting on " ", just make the list itself with individual parameters.
This will allow spaces in paths to be supported properly.
2019-09-10 14:50:55 -07:00
Earle F. Philhower, III
77c4f5e5cf
Add OSX build to CI, fix OSX builds (#6492)
Build a single sketch using a Travis-CI OSX instance to validate the
toolchain works properly on Macs.

Update the installed python3 symlink to point to the proper spot for OSX
(Python3 is in /usr/local/bin, not /usr/bin).

Fixes #6490
2019-09-07 18:31:17 -07:00
david gauchard
5ca0bde200 MDNS: fix random crash on startup (#6261)
* mDNS debug option + AP address is used by default when STA is also present

* mDNS: store network interface, checking it is up

* igmp: force on selected interface (avoid crash *sometimes*)

* fix for all lwip2 ipv4 ipv6 & lwip1

* mdns: IPAddress is not needed to reference associated interface

* mdns: debug: fix print warnings

* emulation: add ets_strncpy

* emulation: truly emulate AddrList (remove fake one)
2019-09-04 20:10:47 -07:00
david gauchard
273f4000f0
Experimental: add new WiFi (pseudo) modes: WIFI_SHUTDOWN & WIFI_RESUME (#6356)
* add new WiFimodes: WIFI_SHUTDOWN & WIFI_RESUME with example
* restore WiFi.onWiFiModeChange()
2019-09-05 03:01:01 +02:00
Earle F. Philhower, III
0a031ce957
Move all scripts and documentation to Python3 (#6378)
* Move all scripts and documentation to Python3

Python 2 EOL is Jan 1, 2020.  Migrate scripts to run under Python 3.

Under Windows, we're already running Python 3.7, by dumb luck.  The
oddness is that the Windows standalone executable for Python 3 is called
"python" whereas under UNIX-like OSes it's called "python3" with
"python" always referring to the Python 2 executable.  The ZIP needs to
be updated to include a Python3.exe (copy of Python.exe) so that we can
use the same command lines under Linux and Windows, and to preserve my
sanity.

Fixes #6376

* Add new Windows ZIP with python3.exe file

* Sort options in boards.txt generation for repeatability

The order of the board opts dict changes depending on the Python version
and machine, so sort the options before printing them to get a stable
ordering.

* Re-add Python2 compatibility tweaks

Most scripts can run as Python 2 or Python 3 with minimal changes, so
re-add (and fix, as necessary) compatibility tweaks to the scripts.
2019-08-28 12:42:48 -07:00
david gauchard
aecf113850 host emulation: littlefs was not fully enabled (#6342) 2019-07-24 18:05:10 -07:00
david gauchard
a64ef544d9
emulation on host: add missing strlcat strlcpy (#6327) 2019-07-23 11:13:50 +02:00
david gauchard
cd6cf984ec
esptool.py: disable 9600bauds in menu for flash upload serial speed (#6292)
* emulation on host: disable coverage when not in CI
* disable 9600bauds for flash upload serial speed (esptool.py won't work)
2019-07-13 23:10:20 +02:00
Earle F. Philhower, III
6bd4b1c4f7
Clean up trivial gcc -wextra warnings (#6254)
After verifying that they really were spurious, clean up the warnings
that gcc -wextra reports, except for LeaMDNS.

Upgrade GCC to gcc-7 for host builds
2019-07-05 22:31:50 -07:00
Earle F. Philhower, III
7036297920 Convert ESP8266WebServer* into templatized model (#5982)
* Convert ESP8266WebServer* into templatized model

Supercedes #4912

Refactor the three versions of ESP8266WebServer and *WebServerSecure to a
single templated class. Use "using" to enable old, non-templated names to b
used (so no user changes required to compile or run).

Fixes #4908 and clean up the code base a lot.

Basic tests run (the ones in the example code).

No code changes are required in userland except for setting the SSL
certificates which now use a cleaner "getServer()" accessor and lets the
app use the native BearSSL calls on the WiFiClientSecure object.

@devyte should be proud, it removes virtuals and even has template specialization...

* Fix HTTPUpdate templates and examples

* Fix HTTPUpdateServer library build

Need to remove dot-a linkage since there are no .cpp files in the
directory anymore due to templates.

* Provide backward-compat names for updt template

Allow existing code to use the same well known names for
HTTPUpdateSecure.

* Remove ClientType from all templates, auto-infer

Remove the ClientType template parameter from all objects.  Simplifies
the code and makes it more foolproof.

Add a "using" in each server to define the type of connection returned
by all servers, which is then used in the above templates automatically.

* Can safely include FS.h now that SD/SPIFFS unified

* Move the templates/objects to their own namespaces

* Fix merge issues with untemplated methods

* Address review comments

* Fix mock test, remove warnings inside test dir

Make the simple mock test CI job pass and clean up
any spurious warnings in the test directory.

There still are warnings in the libraries and core, but they
should be addressed in a separate PR.
2019-07-04 10:58:22 +02:00
david gauchard
6272e897ca
mock: addrList fix (#6248)
* improve mock tcp write
* mock addrlist
* add a single mock build in travis
2019-07-03 09:49:03 +02:00
Earle F. Philhower, III
961b558a91 Fix device test environment variables (#6229)
* Fix device test environment variables

Device tests were not connecting properly to WiFi because the
environment variables were not set when WiFi.connect was called.
This would result in tests sometimes working *if* the prior sketch run
on the ESP saved WiFi connection information and auto-connect was
enabled.  But, in most cases, the tests would simply never connect to
any WiFi and fail.

getenv() works only after BS_RUN is called (because BS_RUN handles the
actual parsing of environment variables sent from the host).

Add a "pretest" function to all tests which is called by the host test
controller only after all environment variables are set.  Move all
WiFi/etc. operations that were in each separate test's setup() into it.

So the order of operations for tests now is:
ESP:  setup()
      -> Set serial baud
      -> Call BS_RUN()
HOST: Send environment
      Send "do pretest"
ESP:  pretest()
      -> Set Wifi using env. ariables, etc. return "true" on success
HOST: Send "run test 1"
ESP:  Run 1st test, return result
HOST: Send "run test 2"
ESP:  Run 2nd test, return result
<and so forth>

If nothing is needed to be set up, just return true from the pretest
function.

All tests now run and at least connect to WiFi.  There still seem to be
some actual test errors, but not because of the WiFi/environment
variables anymore.

* Remove unneeded debug prints

* Silence esptool.py output when not in V=1 mode

Esptool-ck.exe had an option to be silent, but esptool.py doesn't so the
output is very chatty and makes looking a the run logs hard (60 lines
of esptool.py output, 3 lines of actual test reports).

Redirect esptool.py STDOUT to /dev/null unless V=1 to clear this up.

* Speed up builds massively by removing old JSON

arduino-builder checks the build.options.json file and then goes off and
pegs my CPU at 100% for over a minute on each test compile checking if
files have been modified.

Simply deleting any pre-existing options.json file causes this step to
be skipped and a quick, clean recompile is done in siginificantly less
time.

* Enable compile warnings, fix any that show up

Enable all GCC warnings when building the tests and fix any that came up
(mostly signed/unsigned, unused, and deprecated ones).

* Fix UMM_MALLOC printf crash, umm_test

Printf can now handle PROGMEM addresses, so simplify and correct the
debug printouts in umm_info and elsewhere.
2019-06-26 17:54:36 +02:00
david gauchard
7c184f4268 filter weird characters from esp output to python (#6226)
(this is an issue probably since we switched from esptool.exe to esptool.py)
2019-06-25 12:45:33 -07:00
Dirk O. Kaar
653f58e209 interrupts.h in tests/host/common is empty (#6217)
* Per suggestion in https://github.com/esp8266/Arduino/pull/6217#issuecomment-504421729
2019-06-22 16:43:19 +02:00
david gauchard
fc77f2e89c
littlefs: fixes for mock/emulation, use in FSBrowser example (#6211)
* littlefs: fixes for mock/emulation, use in FSBrowser example
* emulation: makefile: integrate arch size into object file names
2019-06-20 23:34:27 +02:00
Ruandv
a78fb72302 Updated Example to use ArduinoJson6 (#6203)
* Updated Example to use ArduinoJson6

* Updated save method to Serialize and not Deserialize

* Updated References to ArduinoJson 6.11.0

* Style Fix

* another line missed

* Added the file extension to the new version
2019-06-20 15:24:33 -04:00
david gauchard
621a341234
emulation: add fake cont_yield (#6210)
* emulation: add fake cont_yield
(temporarily disable littleFS mock due to missing file)

* fix makefile
2019-06-19 19:01:20 +02:00
Earle F. Philhower, III
0920daf251
Add some more CI tests for String::replace (#6193) 2019-06-10 12:51:43 -07:00
Earle F. Philhower, III
78a1a66e6d
Make SSO support \0s, use memmove, add test (#6155)
Supercedes #6027

Make SSO more generic by keeping track of its length explicitly,
allowing for embedded \0s to exist in the String (just like the non-SSO
ones).

Use memmove/memcpy_P when we know the length of a string to save CPU
time.

Add tests to inject \0s in a String to ensure it is still working as
designed.
2019-06-05 08:11:53 -07:00
Earle F. Philhower, III
8859b818d8
Make CertStore natively use File interface (#6131)
__This is a breaking change, but the header and example did warn
everyone that this API was in flux due to the incompatible SD and SPIFFS
File implementations.__

BearSSL CertStores now simply need a filesystem and the names of the
data (generated on-chip) and archive (uploaded by user) files on it.
No more need to roll your own virtual CertStoreFile class.

Update the library, examples, and device test.
2019-05-30 12:53:03 -07:00
Earle F. Philhower, III
a389a995fb Add LittleFS as an optional filesystem, API compatible w/SPIFFS (but not on-flash-format compatible) (#5511)
* Add LittleFS as internal flash filesystem

Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem,
https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard
flash utilizing the exact same API as the existing SPIFFS filesystem.

LittleFS is built for low memory systems that are subject to random power
losses, is actively supported by the ARMmbed community, supports directories,
and seems to be much faster in the large-ish read-mostly applications I use.

LittleFS, however, has a larger minimum file allocation unit and does not do
static wear levelling.  This means that for systems that need many little
files (<4K), have small SPIFFS areas (64K), or which have a large static
set of files covering the majority of flash coupled with a frequently
updated set of other files, it may not perform as well.

Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch,
use LittleFS.open in place of SPIFFS.open to open files, and everything
else just works thanks to the magic of @igrr's File base class.

**LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS**
Since it is a completely different filesystem, you will need to reformat
your flash (and lose any data therein) to use it. Tools to build the
flash filesystem and upload are at
https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and
https://github.com/earlephilhower/mklittlefs/ .  The mklittlefs tool
is installed as part of the Arduino platform installation, automatically.

The included example shows a contrived read-mostly example and
demonstrates how the same calls work on either SPIFFS.* or LittleFS.*
Host tests are also included as part of CI.

Directories are fully supported in LittleFS. This means that LittleFS
will have a slight difference vs. SPIFFS when you use
LittleFS.openDir()/Dir.next().  On SPIFFS dir.next()
will return all filesystem entries, including ones in "subdirs"
(because in SPIFFS there are no subdirs and "/" is the same as any
other character in a filename).

On LittleFS, dir.next() will only return entries in the directory
specified, not subdirs.  So to list files in "/subdir/..." you need
to actually openDir("/subdir") and use Dir.next() to parse through
just those elements.  The returned filenames also only have the
filename returned, not full paths.  So on a FS with "/a/1", "/a/2"
when you do openDir("/a"); dir.next().getName(); you get "1" and "2"
and not "/a/1" and "/a/2" like in SPIFFS.  This is consistent with
POSIX ideas about reading directories and more natural for a FS.

Most code will not be affected by this, but if you depend on
openDir/Dir.next() you need to be aware of it.

Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile,
::openNextFile, and ::rewind methods added to Filesystem objects.
Documentation has been updated with this and other LittleFS information.

Subdirectories are made silently when they do not exist when you
try and create a file in a subdir.  They are silently removed when
the last file in them is deleted.  This is consistent with what
SPIFFS does but is obviously not normal POSIX behavior.  Since there
has never been a "FS.mkdir()" method this is the only way to be
compatible with legacy SPIFFS code.

SPIFFS code has been refactored to pull out common flash_hal_* ops
and placed in its own namespace, like LittleFS.

* Fix up merge blank line issue

* Merge in the FSConfig changs from SDFS PR

Enable setConfig for LittleFS as well plys merge the SPIFFS changes
done in the SDFS PR.

* Fix merge errors

* Update to use v2-alpha branch

The V2-alpha branch supports small file optimizations which can help
increase the utilization of flash when small files are prevalent.
It also adds support for metadata, which means we can start adding
things like file creation times, if desired (not yet).

* V2 of littlefs is now in upstream/master

* Update test to support non-creation-ordered files

In a directory, the order in which "readNextFile()" will return a name
is undefined.  SPIFFS may return it in order, but LittleFS does not as
of V2.  Update the test to look for files by name when doing
readNextFile() testing.

* Fix LittleFS.truncate implementation

* Fix SDFS tests

SDFS, SPIFFS, and LittleFS now all share the same common set of tests,
greatly increasing the SDFS test coverage.

* Update to point to mklittlefs v2

Upgrade mklittlefs to V2 format support

* Remove extra FS::write(const char *s) method

This was removed in #5861 and erroneously re-introduced here.

* Minimize spurious differences from master

* Dramatically reduce memory usage

Reduce the program and read chunk sizes which impacts performance
minimally but reduces per-file RAM usage of 16KB to <1KB.

* Add @d-a-v's host emulation for LittleFS

* Fix SW Serial library version

* Fix free space reporting

Thanks to @TD-er for discovering the issue

* Update littlefs to latest upstream

* Remove sdfat version included by accident

* Update SDFAT to include MOCK changes required

* Update to include SD.h test of file append
2019-05-25 09:53:24 +02:00
david gauchard
b55199227b
add regular scheduled functions, now also callable on yield() (#6039)
* add regular scheduled functions, now also callable on `yield()`

added bool schedule_function_us(std::function<bool(void)> fn, uint32_t repeat_us)
lambda must return true to be not removed from the schedule function list
if repeat_us is 0, then the function is called only once.

Legacy schedule_function() is preserved

This addition allows network drivers like ethernet chips on lwIP to be regularly called
- even if some user code loops on receiving data without getting out from main loop
  (callable from yield())
- without the need to call the driver handling function
  (transparent)

This may be also applicable with common libraries (mDNS, Webserver, )
2019-05-23 22:03:53 +02:00
Earle F. Philhower, III
82adc95d64
Add explicit Print::write(char) (#6101)
W/o this change, file::write('a'); tries to use the template and fails
since char is a basic type.

The reason it is needed is due to pre 2.5.x behavior, File::write(char)
silently was cast to File::write(uint8_t).  With the template write,
though, this is not performed.

* Add Print::write tests and add'l overrides

Ensure that print::write does something sane and doesn't cause a compile
time error about templates when used for Files.

Test using SPIFFS file since Print is an abstract type.
2019-05-18 14:07:13 -07:00
Earle F. Philhower, III
968d6fcb0a
Make FILE_WRITE append in SD.h wrapper (#6106)
* Make FILE_WRITE append in SD.h wrapper

Fixes #6105

* Update host tests
2019-05-16 17:12:09 -07:00
david gauchard
818a55f575 device tests: updates for esptool.py (#6084)
Makefile new rule: 'make distclean' removes virtualenv
attempt to use python3 (python2 is still default)
2019-05-14 08:59:46 -07:00
Allman-astyler
eea9999dc5 Revert "Allman now (#6080)" (#6090)
This reverts commit 98125f88605cd7e46e9be4e1b3ad0600dd5d2b51.
2019-05-14 00:09:54 +02:00
Allman-astyler
98125f8860 Allman now (#6080)
* switch restyle script for CI

* remove confirmation

* restyle with allman
2019-05-13 16:41:34 +02:00
Harald
a67aa569da Test for StreamString SSO bug (#6035) (#6041) 2019-05-03 07:09:32 -07:00
Earle F. Philhower, III
72c21feff6
Make chars unsigned by default, matching ESP8266 (#6013)
Force GCC to run with -funsigned-char during host tests  to make
the PC match the default behaviour used by the xtensa GCC port.

As noted in #6010.  Thanks @MichaelBrunn3r
2019-04-24 08:44:36 -07:00
Michael Brunner
4e0e4e4340 Added notice to update submodules to test README (#6008)
Some users didn't know they had to update the submodules before running tests on host (see esp8266/Arduino/issues/5910). Added a notice to prevent wasting time of users new to git
2019-04-23 15:04:26 +02:00
david gauchard
2c36cfe04d
mock on host: compile without rtti (no dynamic_cast<>(), like on native) (#5994)
* mock on host: compile without rtti (no dynamic_cast<>(), like on native)
2019-04-17 18:27:30 +02:00
Earle F. Philhower, III
885276e75c
Unaligned access support for pgm_read_word/dword (#5692)
* Unaligned access support for pgm_read_word/dword

* Fix pgm_read_ptr_aligned() per #5735

* Allow users to use aligned-only via a #define

Adding -DPGM_READ_UNALIGNED=0 or #define PGM_READ_UNALIGNED 0 will
change the default at compile-time to only aligned (faster, but less
compatible) macro implementations.

Default is still to allow unaligned accesses.
2019-04-11 22:24:49 +03:00
Earle F. Philhower, III
9712170276
Fix String.replace overlapping strcpy (#5966)
* Fix String.replace overlapping strcpy

Fixes #5949

Adds a test from the issue above and fixes the problem valgrind found.

Additional pathological memcpy->memmove fixes
2019-04-10 17:21:15 +03:00
david gauchard
9a2ed274f3 polledTimeout: add option to use CPU count instead of millis() (#5870)
* polledTimeout: add option to use CPU count instead of millis()

* use more "using" alias

* more c++/clear code, using typename (thanks @devyte)

* rename class name to include unit, introduce timeMax() and check it with assert()

* remove useless defines

* improve api readability, add micro-second unit

* update example

* mock: emulate getCycleCount, add/fix polledTimeout CI test

* + nano-seconds, assert -> message, comments, host test

* allow 0 for timeout (enables immediate timeout, fix division by 0)

* typo, set member instead of local variable

* unify error message

* slight change on checkExpired() allows "never expired"
also removed printed message, add YieldAndDelay, simplify calculations

* remove traces of debug.h/cpp in this PR

* include missing <limits> header

* back to original expired test, introduce boolean _neverExpires, fix reset(), getTimeout() is invalid

* fix expiredOneShot with _timeout==0 check

* reenable getTimeout()

* expose checkExpired with unit conversion

* fix timing comments, move critical code to iram

* add member ::neverExpires and use it where relevant

* improve clarity

* remove exposed checkExpired(), adapt LEAmDNS with equivalent

* add API ::resetToNeverExpires(), use it in LEAmDNS

* remove offending constness from ::flagged() LEAmDNS (due do API fix in PolledTimeout)

* simplify "Fast" base classes

* minor variable rename

* Fix examples

* compliance with good c++ manners

* minor changes for consistency

* add missing const

* expired() and bool() moved to iram

* constexpr compensation computing

* add/update comments

* move neverExpires and alwaysExpired
2019-04-05 10:50:53 -03:00
Earle F. Philhower, III
54240d2cc5
Fix String::replace() garbage at end of string (#5897)
* Fix String::replace() 

Fixes #5883 and supercedes #5890

The replace() function was using len() while in the middle of buffer
operations.  In SSO mode len() is not stored separately and is a call to
strlen(), which may not be legal if you're in the middle of overwriting
the SSO buffer, as was the case in ::replace when the replacement string
was longer than the find string.  This caused potential garbage at the
end of the string when accessed.  Instead, just cache the length in a
local while doing the operation.

Add in test cases from #5890 as well as some new ones that fail on the
unmodified core.

* Fix stack smashing error on 64b

When pointers are 8 bytes long, the size of a String is larger than 16
chars.  Increase the allocated array we're using in the test to avoid a
"stack smashing" error.

* Manually call destructor in test

Just for clarity, manually call the destructor for the Strings() that
are "placement new'd" in the String tests.  It is a no-op for the
existing test, since thanks to SSO there are no memory allocations, but
will help in case someone adds tests later which include longer strings.
2019-03-20 06:18:04 -07:00
david gauchard
a3ea816e65
add optional espressif fw nonos-sdk 2.2.x (19.03.13) (#5873) 2019-03-14 13:35:26 +01:00
liebman
e46ccae9d5 optionally allow redirects on HTTPClient & OTA updates (#5009)
* optionally allow redirects on http OTA updates

* Refactored HTTPClient::begin(url...) & setURL functions, now only beginInternal parses URL, sets ports
Added HTTPRedirect example.

* fix indentation for style check

* add space after while for style check

* don't use deprecated begin method in redirect example

* moved redirect handling code to HTTPClient.
only GET and HEAD requests are currently handled automatically
Redirects that fail to be automatically handled return the redirect code as before

* added support for POST/303 redirect
added device redirect tests

* add missing getLocation() implementation

* if the new location is only a path then only update the URI
2019-03-13 01:51:17 +01:00
Earle F. Philhower, III
02f54e85fd
Reimplement SD.h write methods exactly in File (#5861)
* Reimplement SD.h write methods exactly in File

Replace the individual override with the existing SD.h File's
implementation for all methods of File::write.

Fixes #5846

* Add add'l tests

* Fix Print and File incompatible writes w/casts

Print and File have ambiguous resolutions for single-argument
write(0)s.

Fix by adding explicit methods.  A write of any integer will not be a
const char* write (i.e. won't write a string) but will instead just
write the integer truncated to 8 bits (as makes sense).

* Use 256byte chunks in ::write template

Reduce stack requirements for templated writes to 256bytes, matching the
size uses in WiFiClient/etc. (from 512bytes).  Reduces the chance of
stack overflow.

* Move write(int) methods up to Print.h

Remove some technical debt by moving the ::write(int/short/long) methods
out of FS and HardwareSerial and up into Print.h.
2019-03-12 13:58:00 -07:00