* Keep signing commands in platform.txt on release
The boards packager was deleting what was at the time it was written
unused lines in fht platform.txt file before deploying to Arduino.
One of these lines is now needed for signing to work, so don't delete
it.
Also, explicitly call "python signing.py" because it looks like Arduino
is sanitizing/removing executable bits on files when extracting from
boards manager installations.
Fixes#5483
* Create the build subdir, if needed, for autosign
If the temporary build/core directory isn't available, make it in order
that Updater.cpp will see the generated signing header and not the one
in the main core.
* Make exceptions a configurable menu
Add a menu, Exceptions, which allows exceptions to be disabled for ROM
sensitive scripts. Default is enabled.
* Update to latest JSON builder
* Move to PROGMEM aware libc, allow PSTR in printf()
A Newlib (libc) patch is in progress to move the _P functions from inside
Arduino into first-class citizens in libc. This Arduino patch cleans up
code that's been migrated there. Binaries for the new libs are included
because it seems they're part of the Arduino git tree, and should be
replaced with @igrr built ones when/if the Newlib changes are accepted.
Notable changes/additions for Arduino:
Allow for use of PROGMEM based format and parameter strings in all
*printf functions. No need for copying PSTR()s into RAM before printing
them out (transparently saves heap space when using _P functions) and
makes it easier to print out constant strings for applications.
Add "%S" (capital-S) format that I've been told, but cannot verify,
is used in Arduino to specify a PROGMEM string parameter in printfs,
as an alias for "%s" since plain "%s" can now handle PROGMEM.
Optimized the memcpy_P, strnlen_P, and strncpy_P functions to use 32-bit
direct reads whenver possible (source and dest alignment mediated), but
there is still room for improvement in others.
Finally, move several constant arrays from RODATA into PROGMEM and
update their accessors. Among these are the ctype array, ~260 bytes,
mprec* arrays, ~300 bytes, and strings/daycounts in the time
formatting functions, ~200 bytes. All told, sketches will see from
300 to 800 additional RAM heap free on startup (depending on their
use of these routines).
* Fix merge error in #ifdef/#endif
* Fix host test using the newlib generic pgmspace.h
Host tests now use the sys/pgmspace.h for compiles instead of the
ESP8266-specific version.
* Update with rebuilt libraries using latest newlib
* Include binaries built directly from @igrr repo
Rebuild the binaries using a git clone of
https://github.com/igrr/newlib-xtensa
Build commands for posterity:
````
rm -rf ./xtensa-lx106-elf/
./configure --prefix=<DIR>/esp8266/tools/sdk/libc --with-newlib \
--enable-multilib --disable-newlib-io-c99-formats \
--disable-newlib-supplied-syscalls \
--enable-newlib-nano-formatted-io --enable-newlib-reent-small \
--enable-target-optspace \
--program-transform-name="s&^&xtensa-lx106-elf-&" \
--disable-option-checking --with-target-subdir=xtensa-lx106-elf \
--target=xtensa-lx106-elf
rm -f etc/config.cache
CROSS_CFLAGS="-fno-omit-frame-pointer -DSIGNAL_PROVIDED -DABORT_PROVIDED"\
" -DMALLOC_PROVIDED" \
PATH=<DIR>/esp8266/tools/xtensa-lx106-elf/bin/:$PATH \
make all install
````
* Fix merge define conflict in c_types.h
* Fix strlen_P misaligned source error
Include fix from newlib-xtensa/fix-strlen branch cleaning up misaligned
access on a non-aligned source string.
* Fix strlen_P and strcpy_P edge cases
Ran the included test suite on ESP8266 tstring.c with the following defines:
#define MAX_1 50
#define memcmp memcmp_P
#define memcpy memcpy_P
#define memmem memmem_P
#define memchr memchr_P
#define strcat strcat_P
#define strncat strncat_P
#define strcpy strcpy_P
#define strlen strlen_P
#define strnlen strnlen_P
#define strcmp strcmp_P
#define strncmp strncmp_P
Uncovered edge case and return value problems in the optimized versions of
the strnlen_P and strncpy_P functions. Corrected.
* Fix memcpy_P return value
memcpy-1.c test suite showed error in return value of memcpy_P. Correct it.
* Fix strnlen_P/strlen_P off-by-4 error
Random crashes, often on String constructors using a PSTR, would occur due
to the accelerated strnlen_P going past the end of the string. Would make
debug builds fail, too (ESP.getVersionString() failure).
Fix to fall through to normal copy on a word that's got a 0 byte anywhere
in it.
* Add device tests for libc functional verification
Add test suite used to debug libc optimized _P functions to the device
tests.
* Rebuild from igrr's repo (same source as prior)
Rebuild .a from igrr's repo at 347260af117b4177389e69fd4d04169b11d87a97
* WIP - add exceptions
* Fix exception to have 0-terminator
* Move some exception constants to TEXT from RODATA
* Remove throw stubs
* Move more exception stuff to ROM
* Enable exceptions in platform.io
* Remove atexit, is duplicated in rebuilt lib
Need to look at the quick-toolchain options, there seems to be a definition
for atexit defined there (libgcc?) that needs to be excised. For now,
remove our local do-nothing copy.
* Update libgcc to remove soft-fp functions
The esp-quick-toolchain generated libgcc.a needed to have the soft-FP routines
that are in ROM removed from it. Remove them in the new esp-quick-toolchain
and update.
* Fix merge typos in Makefile
* Add unhandled exception handler to postmortem
* Return our atexit() handler
* Latest stdc++, minimize exception emercengy area
* Remove atexit from newlib
atexit was defined in newlib strongly, but we also define a noop atexit in core.
Since we never exit, use the core's noop and delete the atexit from libc.a
Updated in esp-quick-toolchain as well.
* Move __FUNCTION__ static strings to PROGMEM
__FUNCTION__ is unlikely to be a timing sensitive variable, so move it to
PROGMEM and not RODATA (RAM) using linker magic.
asserts() now should take no RAM for any strings.
* Clean up linker file, update to latest stdc++
* Update to latest stdc++ which doesn't call strerror
* Update to GCC5.1 exception emergency allocator
Using GCC 5.1's emergency memory allocator for exceptions, much less
space is required in programs which do not use exceptions and when
space is allocated it is managed more efficiently.
* Initial try with new compiler toolchain
* Include newlib built from esp-quick-toolchain
* Update JSON with all new esp-quick-toolchain builds
* Use 64bit Windows compiler on 64bit Windows
* Dump std::exception.what() when possible
When doing the panic on unhandled exceptions, try and grab the
.what() pointer and dump it as part of the termination info.
Makes it easy to see mem errors (std::bad_alloc) or std::runtime_error
strings.
* Use scripted install from esp-quick-toolchain
Makes sure proper libraries and includes are present by using a
scripted installation from esp-quick-install instead of a manual
one.
* Update eqk to remove atexit, fix packaging diff
Using a pluggable architecture, allow updates delivered via the Update
class to be verified as signed by a certificate. By using plugins, avoid
pulling either axTLS or BearSSL into normal builds.
A signature is appended to a binary image, followed by the size of the
signature as a 32-bit int. The updater takes a verification function
and checks this signature using whatever method it chooses, and if it
fails the update is not applied.
A SHA256 hash class is presently implemented for the signing hash (since
MD5 is a busted algorithm).
A BearSSLPublicKey based verifier is implemented for RSA keys. The
application only needs the Public Key, while to sign you can use
OpenSSL and your private key (which should never leave your control
or be deployed on any endpoints).
An example using automatic signing is included.
Update the docs to show the signing steps and how to use it in the
automatic and manual modes.
Also remove one debugging line from the signing tool.
Saves ~600 bytes when in debug mode by moving strings to PMEM
Windows can't run the signing script, nor does it normally have OpenSSL
installed. When trying to build an automatically signed binary, warn
and don't run the python.
* Move ICACHE_* to unique sections, local LD script
Similar to PROGMEM changes, make the name of each ICACHE_* variable/fuction
unique to avoid issues with section conflicts.
Also rename the generated LD linker script to avoid issue with older copies
of the eagle.app.v6.common.ld which were generated by the build process
in a global directory before being moved to the {build.path}. The linker
would use the older, generated *.ld file instead of the generated one, which
would lead to runtime failures on some systems and cause the VTABLE location
to not correspond to the IDE menu selection.
Fixes#5115, and is an update to #5117 and #5116.
* Update boards.txt.py and platform.io build
arduino-builder 1.3.25 (shipped with Arduino 1.8.5) forces full
recompilation when any file in the core directory is modified. Avoid
full recompilation by placing generated ld script into build
directory, not source directory.
Also fix an issue where git version description would not be generated
if there were spaces in build path.
In tools menu, boards belonging to each platform are grouped together.
The title of the groups is the board package name. When switching
between versions, mentioning the version number in package name helps
identify the installed version.
Rename board group from "ESP8266 Modules" to "ESP8266 Boards", to be
more similar to other platforms, and to reflect the fact that most
entries included are boards, not modules.
As an extra change, when building the boards manager package, set
version number in platform.txt based on git tag name. This saves one
step when creating the release — don't need to update platform.txt
twice.
Closes https://github.com/esp8266/Arduino/issues/5007.
BearSSL (https://www.bearssl.org) is a TLS(SSL) library written by
Thomas Pornin that is optimized for lower-memory embedded systems
like the ESP8266. It supports a wide variety of modern ciphers and
is unique in that it doesn't perform any memory allocations during
operation (which is the unfortunate bane of the current axTLS).
BearSSL is also absolutely focused on security and by default performs
all its security checks on x.509 certificates during the connection
phase (but if you want to be insecure and dangerous, that's possible
too).
While it does support unidirectional SSL buffers, like axTLS,
as implemented the ESP8266 wrappers only support bidirectional
buffers. These bidirectional buffers avoid deadlocks in protocols
which don't have well separated receive and transmit periods.
This patch adds several classes which allow connecting to TLS servers
using this library in almost the same way as axTLS:
BearSSL::WiFiClientSecure - WiFiClient that supports TLS
BearSSL::WiFiServerSecure - WiFiServer supporting TLS and client certs
It also introduces objects for PEM/DER encoded keys and certificates:
BearSSLX509List - x.509 Certificate (list) for general use
BearSSLPrivateKey - RSA or EC private key
BearSSLPublicKey - RSA or EC public key (i.e. from a public website)
Finally, it adds a Certificate Authority store object which lets
BearSSL access a set of trusted CA certificates on SPIFFS to allow it
to verify the identity of any remote site on the Internet, without
requiring RAM except for the single matching certificate.
CertStoreSPIFFSBearSSL - Certificate store utility
Client certificates are supported for the BearSSL::WiFiClientSecure, and
what's more the BearSSL::WiFiServerSecure can also *require* remote clients
to have a trusted certificate signed by a specific CA (or yourself with
self-signing CAs).
Maximum Fragment Length Negotiation probing and usage are supported, but
be aware that most sites on the Internet don't support it yet. When
available, you can reduce the memory footprint of the SSL client or server
dramatically (i.e. down to 2-8KB vs. the ~22KB required for a full 16K
receive fragment and 512b send fragment). You can also manually set a
smaller fragment size and guarantee at your protocol level all data will
fit within it.
Examples are included to show the usage of these new features.
axTLS has been moved to its own namespace, "axtls". A default "using"
clause allows existing apps to run using axTLS without any changes.
The BearSSL::WiFi{client,server}Secure implements the axTLS
client/server API which lets many end user applications take advantage
of BearSSL with few or no changes.
The BearSSL static library used presently is stored at
https://github.com/earlephilhower/bearssl-esp8266 and can be built
using the standard ESP8266 toolchain.
* Add flash for vtable destination, make it default
Add an option for placing vtables in flash to complement the existing
iram and heap options. "make flash"
Now that there is a way to change it, move to vtables in flash as default
as only users with interrupts which use vtables require the vtable to
be in RAM. For those users, if the tables are small enough they can put
them in IRAM and save heap space for their app. If not, then the vtables
can be placed in HEAP which supports much larger tables.
* Add VTable menu, FLASH as default, remove Makefile
Convert from manual "make" operated app.ld creation to runtime creation
whose options are selected from the build menu.
Use a prelink recipe to create the output app.ld file each run, without
need for any special tools.
Update the boards.txt.py script to generate this new config.
Generate and preserve a linker .MAP file for the project in the usual
build directory. This map file is useful for finding out where FLASH,
IROM, and IRAM are being used as it shows gcc internally generated
things such as jump tables and constant function-local variables.
* provide full version descriptor, displayed in debug mode
* unix: shows core version like under windows when git is unavailable
* store strings in progmem
* version string honours NDEBUG
* add ARDUINO_ESP8266_GIT_DESC
restore ARDUINO_ESP8266_GIT_VER
restore global variable "core_version"
don't print full version on setDebugOutput(true)
set platform.txt version to 2.4.1-pre
hide irrelevant boot version
fix typo
* lwip2: fix disconnection/reconnection issue
also:
improve version string
remove useless message
* lwip2: bump tag before 2.4.1
* lwip2: improve netif flags management on git side
* full-version string: remove useless NDEBUG in separate source file
* do not automatically enable sdk messages along with core messages
* automatically reenable sdk messages along with core messages *before* setup not after
* check serial port when showing version-string + move sdk messages enabler in hardware serial
* + license header
* updated and tested windows commands in platform.txt (without git)
* updated and tested windows commands in platform.txt (without git)
* update package builder accordingly
+ generates boards.rst
+ generate and replace boards section in package.json
+ generate ldscripts
+ new debug option: OOM
+ new led menu for generic board
- Update SDK header files and libraries to SDK 2.0.0 plus 2.0.0_16_08_09
patch
- Remove mem_manager.o from libmain.a (replaced with umm_malloc)
- Disable switch from DIO to QIO mode for certain flash chips (saves
IRAM space)
- Add user_rf_cal_sector_set; it points to rf_init_data sector.
- Change the way rf_init_data is spoofed.
This is now done by wrapping spi_flash_read and returning the data we
need during startup sequence.
- Place lwip library into flash using linker script instead of section
attributes (saves IRAM space)
* Use newlib libc library
This change adds libcmin.a, which is created from newlib libc by selectively removing some of the object files (mostly related to heap management).
The list of files is available in tools/sdk/lib/make_libcmin.sh. Files which are not needed are commented out.
This change adds support for various functions which were missing, like sscanf, strftime, etc.
* Fix some of the time functions
* Redirect stdout to serial
* Implement __putc_r
* Switch to custom newlib build
Built from https://github.com/igrr/newlib-xtensa using:
./configure --with-newlib --enable-multilib --disable-newlib-io-c99-formats --enable-newlib-supplied-syscalls --enable-target-optspace --program-transform-name="s&^&xtensa-lx106-elf-&" --disable-option-checking --with-target-subdir=xtensa-lx106-elf --target=xtensa-lx106-elf --enable-newlib-nano-formatted-io --enable-newlib-reent-small --prefix=path-to-arduino-core/tools/sdk/libc
CROSS_CFLAGS="-DMALLOC_PROVIDED -DSIGNAL_PROVIDED -DABORT_PROVIDED" make
make install
* Update tests
