* Clean up minor warnings from LGTM.com
LGTM (Semmie) is a tool, bought by GitHub last year, that conducts basic
linting tasks on code and HTML.
Clean up the warnings identified in the latest report:
https://lgtm.com/projects/g/esp8266/Arduino/?mode=list
No functionality should change, however this may fix some issues with
the perl utilities not exiting properly on a Ctrl-C from the command
line.
* Back out HTML changes and rerun boards.txt.py
* remove lwip-v1.4 specific code
* ditto
* ditto
* fix ip4_addr definition
* CI: change debug builds to use IPv6, remove regular IPv6 builds
* ditto
* split pio CI in four (because they last twice the time of the other builds)
* remove option from pio
* remove lwIP-1.4 from doc
* restore pio CI splitting
* fix CI debug6 script
* ditto
* Upgrade to GCC 9.1 toolchain
* Rebuilt using pure GNU binutils and GCC
Remove dependencies on earlier forked GNU utilities (gcc-xtensa,
binutils-gdb-xtensa) and just use GCC sources, unmodified (except for
patches in the esp-quick-toolchain directories).
* Rebuild bearssl using new toolchain
* Fix GDBstub linkage options
GDB works with pure GNU GCC and pure GNU binutils now. Still warnings
galore, but tested with the example sketch in the docs.
* Fix digitalRead alias warning
* Remove gdb stub warnings w/a pragma
* Fix deprecated implicit copy ctors in IP code
Fix some warnings present in GCC8/9 in the IPAddress code
In AddressListIterator there was a copy constructor which simply copied
the structure bit-for-bit. That's the default operation, so remove it
to avoid the warning there.
IPAddress, add a default copy constructor since the other copy
constructors are simply parsing from one format into a native ip_addr_t.
@d-a-v, can you give these a look over and see if they're good (since
IP stuff is really your domain).
* Fix AxTLS alias function defs to match real code
* Fix WiFiClientSecure implicit default copy ctor
These both use shared-ptrs to handle refcnts to allocated data, so using
the default copy constructor is fine (and has been in use for a long
time).
* Dummy size for heap to avoid GCC 8/9 warnings
Make GCC think _heap_start is large enough to avoid the basic (and
incorrect) bounds-checking warnings it produces. The size chosen is
arbitrary and does not affect the actual size of the heap in any way.
* Make heap an undefined extend array
Instead of a bogus size, use an indefinite size for the heap to avoid
GCC warnings
* Trivial tab to space fix
* Update SDFat to remove FatFile warnings
* Fix ticker function cast warnings in GCC 9
The callback function is defined to take a (void*) as parameter, but our
templates let users use anything that fits inside sizeof(void*) to be
passed in. Add pragmas to stop GCC warnings about this, since we
already check the size of the type will fit in the allocated space.
* Remove GCC support fcn that's in ROM
Manually delete the divdi3.so from the libgcc.a library by running the
updated EQT's 9.1-post script.
* Make exceptions work again, get std::regex up
Exceptions are broken on all builds (GCC4.8-9.1) due to the removal of
the PROGMEM non-32b read exception handler (added in the unstable
pre3.0.0).
Build the exception code with -mforce-l32 and patch
accordingly to avoid LoadStore errors.
Apply patches to select portions of the regex lib which use _stype_
(which is now in flash).
* Rebuild Bearssl using latest GCC push
* Automate building of BearSSL and LWIP w/new toolchain
* Workaround g++ template section problem for exception strings
G++ seems to throw out the section attributes for templates. This means
that the __EXCSTR(a synonym for "PSTR()") is ignored and exception.what
strings are stored in RODATA, eating up RAM.
Workaround by using the linker to place the strings keying off their name
("*__exception_what__*").
* Rebuild moving exception.what to unique names
Exception.whats are now all in __exception_what__ and can be moved by
the linker to flash. Works aroung G++ issue with segments being lost in
templates.
* Rebuild with new LWIP locking
* Update to latest libs, save iram
Move two GCC FP support routines out of iram since they are in ROM
already, saving some add'l IRAM. Same list as gcc 4.8.
* Update BearSSL to latest release
* Fix umm_perf reference to ROM function
* Fix "reinterpret_case is not a constexpr" error
In GCC 9 (and 8 from what I read on SO), a cast of a const int to a
function pointer (via explicit or implicit reinterpret_cast) is not a
constexpr.
````
/home/earle/Arduino/hardware/esp8266com/esp8266/cores/esp8266/umm_malloc/umm_performance.cpp:45:36: error: a reinterpret_cast is not a constant expression
45 | int constexpr (*_rom_putc1)(int) = (int (*)(int))(void*)0x40001dcc;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
````
Remove the constexpr, potentially increasing heap usage by 4 bytes in
debug mode.
* Update libc.a to latest rev
* Full rebuild of toolchain and libs
* Upgrade to GCC 9.2, released in August 2019
Add builds for all 5 cross-compiles to pass CI
* Move to --std=gnu++14 (C++14 + GNU extensions)
* Fix Ticker merge conflict
* Fix json merge conflict
* One more merge diff fix
* Reapply Ticker.h casting warning fixes for gcc9
* Update with fixes from Sming repo for PSTR and ASM
* Upgrade to -gnu4 toolchain
* Move to gnu5 build with add'l softFP from ROM
* Move add'l softFP from IRAM to flash
Per @mikee47, we miss sone add'l soft-FP routined in the linker which
makes them end up in IRAM. Move them explicitly into flash, like a
couple others we have already done this for.
* Move to std=c++17/c17 in platform, remove abs/round
Move to C++17 and C17 standards on the compiler options.
Remove "register" from core since it is deprecated.
Remove the #define abs() and #define round() which a) overwrote the
C stdlib definitions, poorly, and b) broke the GCC core code which used
"abs" as an internal function name.
Now abs() returns an integer, and not whatever was being absoluted.
fabs() etc. can be used if users need fload/double suport.
round() returns a double now, which is basically what it was returning
in the original case since adding/subtracting by a FP.
* Use std::abs/round to replace the macro definitions
Per discussion w/@devyte, preserve the abs() and round() functionality
via the using statement.
* Remove using std::abs which conflicted with C lib headers
* Add 2nd arg (exception handler) to ets_isr_t
Disassembly of the ROM shows there are 2 params to the ets_isr_t
callback. The first is the arg passed in, the second is a pointer to an
exception frame where you can get info about when the IRQ happened.
* Move the gdbstub example to a subdir
The Arduino IDE and the build CI don't build it without a subdir, so
make one for gdbstub's example so it's visible and tested.
* Fix ets_irq_arratch redefinition and core IRQ handlers
Remove a duplicated, different declaration for ets_irq_attach from
ets_sys.h. It never really even matched the other declaration in the
same header.
Update the core to IRQ handlers to fix the prototype and include the
2nd, unused frame parameter.
* Actually rebuild the libc.a using GCC 9.2
* Fix SPISlave interrupt attach's 2nd parameter
* Rebuild eboot.elf with GCC 9
* Update to latest SoftwareSerial for Delegate fix
* Upgrade to GCC 9.3
* Rebuild all arch toolchains
* Move to GCC 10.1
* Merge master and fix eboot build
GCC10 now uses `-fno-common` so the eboot global variables were being
placed in IRAM. Adjust the makefile and rebuild to fix.
* Built complete toolchain for all archs
* Pull in latest PSTR changes and fix GCC10.1 build
Somehow the prior GCC build's -mforce32 patch wasn't applying correctly,
but I was still able to get a binary. Fixed. Also pulled in latest
PSTR changes in progmem.h
* Update platform.io to platform C/C++ standards
* Use PR's toolchain in platformio build
* Fix several asm warnings in PIO build
* Optional stack smash protection -fstack-protector
Add a menu to enable GCC's built-in stack smash protection. When a
subroutine goes past its end of stack, generate a crashdump on function
exit like:
````
GCC detected stack overrun
Stack corrupted, stack smash detected.
>>>stack>>>
ctx: cont
sp: 3fffff20 end: 3fffffc0 offset: 0000
3fffff20: 40202955 00000001 0000001c 4020287e
3fffff30: feefeffe 000000fd 00000000 00000000
...
<<<stack<<<
````
Disabled by default because there is a small per-function code overhead
(and CPU time if the function is called very frequently and is very
small).
BearSSL and LWIP are not built using stack smash detection, yet.
* Fix duplicated stc=gnu99/c17 in build
* Dump faulting function PC in stack overflow
Report a fake exception to have the exception decoder print the actual
faulting function. This won't tell you where in the function the issue
happened, but it will tell you the function name first and foremost.
* Rebuild with Platform.io JSON tag in release tgzs
* Deprecate SPIFFS, move examples to LittleFS
SPIFFS has been a great filesystem, but it has significant problems in
many cases (and it's also pretty slow). Development seems to have
slowed/stopped on the upstream version, and we're not able to provide
support or fix the known issues with it as-is.
Deprecate SPIFFS variable.
Update all examples to use LittleFS instead of SPIFFS.
Also, minor cleanup on very old examples which has obsolete delays
waiting for the Serial port to come up, or which were stuck at 9600 baud
because of their ancient AVR heritage.
Fixes#7095
* Remove leftover debug code
* Clean up comments in some examples
* Update documentation on SPIFFS deprecation
* Fix host tests to avoid deprecation warnings
* Fix cut-n-paste error
* Restore SpeedTest.ino, adjust to allow custom FSes
Co-authored-by: Develo <deveyes@gmail.com>
as proposed by https://github.com/esp8266/Arduino/pull/6972#issue-358508056.
@ttytyper 's changes have been incorporate into this PR
The build flag ARDUINO_SONOFF_... should now appear as ARDUINO_ESP8266_SONOFF_...
@ttytyper, @mcspr, and @d-a-v thanks!
* sdk:22x191122
another firmware to test
not default: v2.2.1-119-ga0b1311 (shows as SDK:2.2.2-dev(a58da79) in debug mode)
default unchanged
* and the obj files
* Add boards filter support - allows for the creation of an abridged boards.txt.
Add some of the ITEAD Sonoff boards to boards.txt.py.
Minor reorder of presentation of board menu items, mainly grouped
board Model and module selection to the top.
* Corrected, I think, LED_BUILTIN vs BUILTIN_LED??
* Updated boards.txt
* Added support for DOIT ESP-Mx DevKit (ESP8285) board.
Adjusted wording and fixed side bar formating issue on the
Sonoff description. Also, removed resetmethod menu
and assigned resetmethod of none.
* FW: use NONOS-SDK branch 2.2.x from 2019-10-24 by default
former one is available when using generic board configuration
* tv qvoqve, Platform-ii
* nonos-sdk v2.2.x from 2019-11-05 by default for all boards
(previous ones can be selected with the generic board)
* (w/ fw files)
* 191024 by default, 191105 is an option
* Reset method changes for esptool.py
* Workaround - to be reverted in case esptool do erase/write_flash in one command
* Keep previous resetmethod names and translate to esptool.py options
* Regenerated boards.txt
* adds initial commit to boards.txt before actual edits
* reminder to remove vim .swp file
* Removes spaces and modifies outdated tags for menu `CpuFrequency -> xtal` for example
* Adds SparkFun Bynk Board to boards.txt.py, moves .build flags all together for SparkFun Blynk
* runs boards.txt.py to generate files for pull request - all files have been included as instructed at top of boards.txt file
* Deletes three .orig files generated by boards.txt.py: rst, txt, and json
* Moves boards.txt.py back to /tools directory and attempts to change its mode.
* restore 'x' flag
* sdk: testing branch update (v2.2.x), tidy up fw names in menus, add dates
Former default SDK (22y, renamed to 22x-190703) is still available
Changelog is included in sdk directory
* Keep current FW as default until next release
* update for PIO
In order to give user libs a change to update to the new symbols, re-add
the _SPIFFS_XX symbols to the linker file with a comment that they are
deprecated.
Also add back spiffs_hal_xxx functions, also marked as deprecated.
Fixes#6542
A number of non-genuine boards exist mainly from flea-bay sellers that
use under-sized and/or low quality flash chips which can not handle
a 40MHz FlashFreq properly.
This patch adds slower flash frequencies to the menu for generic ESP boards
so that these cheap knock-offs can be run in a stable manner, hopefully saving
some people a few headaches and keeping these boards out of landfill.
When the FS_END was adjusted to end on a full block (i.e. rounded down)
to avoid filesystem issues, but _FS_end was changed. The EEPROM library
used _FS_end to implicitly calculate the start of the EEPROM data, so
this means after the _FS_end fix, EEPROM data written with prior
releases would "disappear."
Avoid the issue by explicitly calculating the EEPROM start location in
the linker, using the same formula as prior release.
Fixes#6531
* mDNS debug option + AP address is used by default when STA is also present
* mDNS: store network interface, checking it is up
* igmp: force on selected interface (avoid crash *sometimes*)
* fix for all lwip2 ipv4 ipv6 & lwip1
* mdns: IPAddress is not needed to reference associated interface
* mdns: debug: fix print warnings
* emulation: add ets_strncpy
* emulation: truly emulate AddrList (remove fake one)
* Move all scripts and documentation to Python3
Python 2 EOL is Jan 1, 2020. Migrate scripts to run under Python 3.
Under Windows, we're already running Python 3.7, by dumb luck. The
oddness is that the Windows standalone executable for Python 3 is called
"python" whereas under UNIX-like OSes it's called "python3" with
"python" always referring to the Python 2 executable. The ZIP needs to
be updated to include a Python3.exe (copy of Python.exe) so that we can
use the same command lines under Linux and Windows, and to preserve my
sanity.
Fixes#6376
* Add new Windows ZIP with python3.exe file
* Sort options in boards.txt generation for repeatability
The order of the board opts dict changes depending on the Python version
and machine, so sort the options before printing them to get a stable
ordering.
* Re-add Python2 compatibility tweaks
Most scripts can run as Python 2 or Python 3 with minimal changes, so
re-add (and fix, as necessary) compatibility tweaks to the scripts.
* enable by default latest 2.2.x firmware, including fixed espnow
* LittleFS: avoid crash when FS size is 0
* flash size defaults: 1M for generic board, not empty FS for all
* Make SPIFFS be an integer number of blocks
boards.txt.py simply calculated the end and start using flash sizes, but
in cases where an 8K page was used (>512KB SPIFFS), this could leave a
4K half-block left at the end of SPIFFS.
mkspiffs and the SPIFFS code uses integer division to calculate the
maximum block number, so it worked fine in practice and the code simply
ignored the extra, fractional block.
Now actually take block size into account when calculating the end of
SPIFFS, ensuring no fractional blocks are passed in. Does not result in
data loss on pre-existing SPIFFS filesystems.
* Fix the 1m512 case and clean up code
Ensure that no SPIFFS_block in the LD files is modified from the
original to endure correct backwards compatibility
* Factor out common if, clean code
* Make boards.py vars "fs_xx" instead of "spiffs_xx"
* Add LittleFS as internal flash filesystem
Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem,
https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard
flash utilizing the exact same API as the existing SPIFFS filesystem.
LittleFS is built for low memory systems that are subject to random power
losses, is actively supported by the ARMmbed community, supports directories,
and seems to be much faster in the large-ish read-mostly applications I use.
LittleFS, however, has a larger minimum file allocation unit and does not do
static wear levelling. This means that for systems that need many little
files (<4K), have small SPIFFS areas (64K), or which have a large static
set of files covering the majority of flash coupled with a frequently
updated set of other files, it may not perform as well.
Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch,
use LittleFS.open in place of SPIFFS.open to open files, and everything
else just works thanks to the magic of @igrr's File base class.
**LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS**
Since it is a completely different filesystem, you will need to reformat
your flash (and lose any data therein) to use it. Tools to build the
flash filesystem and upload are at
https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and
https://github.com/earlephilhower/mklittlefs/ . The mklittlefs tool
is installed as part of the Arduino platform installation, automatically.
The included example shows a contrived read-mostly example and
demonstrates how the same calls work on either SPIFFS.* or LittleFS.*
Host tests are also included as part of CI.
Directories are fully supported in LittleFS. This means that LittleFS
will have a slight difference vs. SPIFFS when you use
LittleFS.openDir()/Dir.next(). On SPIFFS dir.next()
will return all filesystem entries, including ones in "subdirs"
(because in SPIFFS there are no subdirs and "/" is the same as any
other character in a filename).
On LittleFS, dir.next() will only return entries in the directory
specified, not subdirs. So to list files in "/subdir/..." you need
to actually openDir("/subdir") and use Dir.next() to parse through
just those elements. The returned filenames also only have the
filename returned, not full paths. So on a FS with "/a/1", "/a/2"
when you do openDir("/a"); dir.next().getName(); you get "1" and "2"
and not "/a/1" and "/a/2" like in SPIFFS. This is consistent with
POSIX ideas about reading directories and more natural for a FS.
Most code will not be affected by this, but if you depend on
openDir/Dir.next() you need to be aware of it.
Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile,
::openNextFile, and ::rewind methods added to Filesystem objects.
Documentation has been updated with this and other LittleFS information.
Subdirectories are made silently when they do not exist when you
try and create a file in a subdir. They are silently removed when
the last file in them is deleted. This is consistent with what
SPIFFS does but is obviously not normal POSIX behavior. Since there
has never been a "FS.mkdir()" method this is the only way to be
compatible with legacy SPIFFS code.
SPIFFS code has been refactored to pull out common flash_hal_* ops
and placed in its own namespace, like LittleFS.
* Fix up merge blank line issue
* Merge in the FSConfig changs from SDFS PR
Enable setConfig for LittleFS as well plys merge the SPIFFS changes
done in the SDFS PR.
* Fix merge errors
* Update to use v2-alpha branch
The V2-alpha branch supports small file optimizations which can help
increase the utilization of flash when small files are prevalent.
It also adds support for metadata, which means we can start adding
things like file creation times, if desired (not yet).
* V2 of littlefs is now in upstream/master
* Update test to support non-creation-ordered files
In a directory, the order in which "readNextFile()" will return a name
is undefined. SPIFFS may return it in order, but LittleFS does not as
of V2. Update the test to look for files by name when doing
readNextFile() testing.
* Fix LittleFS.truncate implementation
* Fix SDFS tests
SDFS, SPIFFS, and LittleFS now all share the same common set of tests,
greatly increasing the SDFS test coverage.
* Update to point to mklittlefs v2
Upgrade mklittlefs to V2 format support
* Remove extra FS::write(const char *s) method
This was removed in #5861 and erroneously re-introduced here.
* Minimize spurious differences from master
* Dramatically reduce memory usage
Reduce the program and read chunk sizes which impacts performance
minimally but reduces per-file RAM usage of 16KB to <1KB.
* Add @d-a-v's host emulation for LittleFS
* Fix SW Serial library version
* Fix free space reporting
Thanks to @TD-er for discovering the issue
* Update littlefs to latest upstream
* Remove sdfat version included by accident
* Update SDFAT to include MOCK changes required
* Update to include SD.h test of file append
* New menu option to minimize BSSL ROM with only RSA
Adds a menu option and define to limit BearSSL to older RSA connection
options. This saves ~45K program memory and can speed up connections
since EC, while more secure, is significantly slower on the chip.
The supported ciphers are identical to the ones that axTLS supported.
Fixes#6005
* Add default SSLFLAGS(blank) to platform.txt
* Fix unused variable warning
* Add clarifying comment to menu items
This commit allows switching SDK firmware:
nonos-sdk-pre-v3 shipped with release 2.5.0 has issues:
* Some boards show erratic behavior (radio connection is quickly lost), with an unknown cause.
These boards work well with previous nonos-sdk-2.2.1 firmware (#5736)
* Overall performances seem to have decreased (#5513)
This PR restores sdk2.2.1 (as in core-2.4.2).
SDK-pre-3.0 - which has brought long awaited fixes (WiFi sleep modes) - is still available through a menu option available only with generic board.
BREAKING
* new define `-DNONOSDK221=1` or `-DNONOSDK3V0=1`
* for external build systems: new library directory: `tools/sdk/lib/<version>/lib`
* PIO: variable `PIO_FRAMEWORK_ARDUINO_ESPRESSIF_SDK3` is needed for sdk-pre-v3.
Fix#5736
* Add esptool.py, pyserial, and python to JSON
Add installation of python on Win32/Win64, and on all systems install
esptool.py and pyserial.
* Initial esptool.py upload test
* First successfull esptool.py upload
* Patch in verbose flag operation
* Replace esptool-ck.exe with Python equivalent
Remove need for binary esptool-ck.exe by implementing the same logic as
esptool-ck uses in Python. Only image creation is supported, and only
in the Arduino standard mode (with its custom bootloader and ROM
layout).
* Remove all esptool-ck.exe, hook Windows Python
Remove all references to esptool-ck and use Python on Windows and Linux
for all recipes where possible.
* Use python to make core_version as well
Avoid ugly bash and CMD.exe tricks in platform.txt by using python to
make the core_version header.
* Rename conflicting script, clean up packager
* Windows test passes
Need to make sure Python2 and Python3 compatible and paths are munged
properly to avoid eaccidentally escaping things when calling esptool.py
Able to compile, build a BIN and upload via esptool.py on a Windows
machine without Python installed globally, only as part of the Arduino
tools package.
* Use github sources for pyserial
* Erase calibration or all flash before programming
Add back in erase support by calling esptool.py twice (since it does not
support chained operations like esptool-ck.exe).
* Make 460K default speed, remove 961K
961K doesn't seem to work with esptool, so make 460K the default upload
speed and remove 961K.
Even at this lower speed, esptool.py is much faster to upload (even
before taking into account the compression when doing things like SPIFFS
and code upload).
* Make erase and upload work again
Arduino does not support a upload.#.cmd pattern, so we need to do
everything in a single command line. Make it cleaner by introducing a
Python wrapper script which will run the same executable with different
sets of commands (since we need to erase a block w/a separate invocation
from the real upload).
Update boards.txt to use the new options format, placing the esptool
command as "version" when there is no "erase_flash" or "erase_region" to
be done to keep things simple.
* Move esptool/pyserial to submodules
Since esptool.py and pyserial are coming directly from github repos,
there is no need to include them as a tool in package.json.
* Restore 921K upload opt, silent downgrade to 460k
To enable full backward compatibility, restore the 921k option for
upload speed but silently change it to 460k in the upload.py script.
Add error checking on upload.py
Scripts, makefiles, and users who do no changes will not have exceptions
enabled during builds. This should avoid the sketch inflation issue for
users who are space constrained, while allowing users who care about
exceptions to enable them through the IDE.
* Make exceptions a configurable menu
Add a menu, Exceptions, which allows exceptions to be disabled for ROM
sensitive scripts. Default is enabled.
* Update to latest JSON builder
The packages JSON file which includes the boards, tools, etc. and needs to
have consistent formatting to be reproducible. The current boards.txt.py
uses a REGEX to string-replace a bit of it, but that bit has a different
indent than the rest of the file.
Use Python's JSON writer to format the whole file repeatably.
