1
0
mirror of https://github.com/esp8266/Arduino.git synced 2025-06-22 08:22:04 +03:00

Enable exceptions, update to optimized newlib, migrate to new toolchain (#5376)

* Move to PROGMEM aware libc, allow PSTR in printf()

A Newlib (libc) patch is in progress to move the _P functions from inside
Arduino into first-class citizens in libc.  This Arduino patch cleans up
code that's been migrated there.  Binaries for the new libs are included
because it seems they're part of the Arduino git tree, and should be
replaced with @igrr built ones when/if the Newlib changes are accepted.

Notable changes/additions for Arduino:
Allow for use of PROGMEM based format and parameter strings in all
*printf functions.  No need for copying PSTR()s into RAM before printing
them out (transparently saves heap space when using _P functions) and
makes it easier to print out constant strings for applications.

Add "%S" (capital-S) format that I've been told, but cannot verify,
is used in Arduino to specify a PROGMEM string parameter in printfs,
as an alias for "%s" since plain "%s" can now handle PROGMEM.

Optimized the memcpy_P, strnlen_P, and strncpy_P functions to use 32-bit
direct reads whenver possible (source and dest alignment mediated), but
there is still room for improvement in others.

Finally, move several constant arrays from RODATA into PROGMEM and
update their accessors.  Among these are the ctype array, ~260 bytes,
mprec* arrays, ~300 bytes, and strings/daycounts in the time
formatting functions, ~200 bytes.  All told, sketches will see from
300 to 800 additional RAM heap free on startup (depending on their
use of these routines).

* Fix merge error in #ifdef/#endif

* Fix host test using the newlib generic pgmspace.h

Host tests now use the sys/pgmspace.h for compiles instead of the
ESP8266-specific version.

* Update with rebuilt libraries using latest newlib

* Include binaries built directly from @igrr repo

Rebuild the binaries using a git clone of
https://github.com/igrr/newlib-xtensa

Build commands for posterity:
````
rm -rf ./xtensa-lx106-elf/
./configure --prefix=<DIR>/esp8266/tools/sdk/libc --with-newlib \
            --enable-multilib --disable-newlib-io-c99-formats \
            --disable-newlib-supplied-syscalls \
            --enable-newlib-nano-formatted-io --enable-newlib-reent-small \
            --enable-target-optspace \
            --program-transform-name="s&^&xtensa-lx106-elf-&" \
            --disable-option-checking --with-target-subdir=xtensa-lx106-elf \
            --target=xtensa-lx106-elf
rm -f etc/config.cache
CROSS_CFLAGS="-fno-omit-frame-pointer -DSIGNAL_PROVIDED -DABORT_PROVIDED"\
             " -DMALLOC_PROVIDED" \
  PATH=<DIR>/esp8266/tools/xtensa-lx106-elf/bin/:$PATH \
  make all install
````

* Fix merge define conflict in c_types.h

* Fix strlen_P misaligned source error

Include fix from newlib-xtensa/fix-strlen branch cleaning up misaligned
access on a non-aligned source string.

* Fix strlen_P and strcpy_P edge cases

Ran the included test suite on ESP8266 tstring.c with the following defines:
 #define MAX_1 50
 #define memcmp memcmp_P
 #define memcpy memcpy_P
 #define memmem memmem_P
 #define memchr memchr_P
 #define strcat strcat_P
 #define strncat strncat_P
 #define strcpy strcpy_P
 #define strlen strlen_P
 #define strnlen strnlen_P
 #define strcmp strcmp_P
 #define strncmp strncmp_P

Uncovered edge case and return value problems in the optimized versions of
the strnlen_P and strncpy_P functions.  Corrected.

* Fix memcpy_P return value

memcpy-1.c test suite showed error in return value of memcpy_P.  Correct it.

* Fix strnlen_P/strlen_P off-by-4 error

Random crashes, often on String constructors using a PSTR, would occur due
to the accelerated strnlen_P going past the end of the string. Would make
debug builds fail, too (ESP.getVersionString() failure).

Fix to fall through to normal copy on a word that's got a 0 byte anywhere
in it.

* Add device tests for libc functional verification

Add test suite used to debug libc optimized _P functions to the device
tests.

* Rebuild from igrr's repo (same source as prior)

Rebuild .a from igrr's repo at 347260af117b4177389e69fd4d04169b11d87a97

* WIP - add exceptions

* Fix exception to have 0-terminator

* Move some exception constants to TEXT from RODATA

* Remove throw stubs

* Move more exception stuff to ROM

* Enable exceptions in platform.io

* Remove atexit, is duplicated in rebuilt lib

Need to look at the quick-toolchain options, there seems to be a definition
for atexit defined there (libgcc?) that needs to be excised.  For now,
remove our local do-nothing copy.

* Update libgcc to remove soft-fp functions

The esp-quick-toolchain generated libgcc.a needed to have the soft-FP routines
that are in ROM removed from it.  Remove them in the new esp-quick-toolchain
and update.

* Fix merge typos in Makefile

* Add unhandled exception handler to postmortem

* Return our atexit() handler

* Latest stdc++, minimize exception emercengy area

* Remove atexit from newlib

atexit was defined in newlib strongly, but we also define a noop atexit in core.
Since we never exit, use the core's noop and delete the atexit from libc.a

Updated in esp-quick-toolchain as well.

* Move __FUNCTION__ static strings to PROGMEM

__FUNCTION__ is unlikely to be a timing sensitive variable, so move it to
PROGMEM and not RODATA (RAM) using linker magic.

asserts() now should take no RAM for any strings.

* Clean up linker file, update to latest stdc++

* Update to latest stdc++ which doesn't call strerror

* Update to GCC5.1 exception emergency allocator

Using GCC 5.1's emergency memory allocator for exceptions, much less
space is required in programs which do not use exceptions and when
space is allocated it is managed more efficiently.

* Initial try with new compiler toolchain

* Include newlib built from esp-quick-toolchain

* Update JSON with all new esp-quick-toolchain builds

* Use 64bit Windows compiler on 64bit Windows

* Dump std::exception.what() when possible

When doing the panic on unhandled exceptions, try and grab the
.what() pointer and dump it as part of the termination info.
Makes it easy to see mem errors (std::bad_alloc) or std::runtime_error
strings.

* Use scripted install from esp-quick-toolchain

Makes sure proper libraries and includes are present by using a
scripted installation from esp-quick-install instead of a manual
one.

* Update eqk to remove atexit, fix packaging diff
This commit is contained in:
Earle F. Philhower, III
2018-12-02 22:37:14 -08:00
committed by Develo
parent 4941711505
commit 6280e98b03
37 changed files with 2273 additions and 811 deletions

View File

@ -254,7 +254,7 @@ const int TIM_DIV265 __attribute__((deprecated, weak)) = TIM_DIV256;
#ifdef __cplusplus
#include <algorithm>
#include "pgmspace.h"
#include <pgmspace.h>
#include "WCharacter.h"
#include "WString.h"

View File

@ -28,36 +28,6 @@ using __cxxabiv1::__guard;
extern void *umm_last_fail_alloc_addr;
extern int umm_last_fail_alloc_size;
void *operator new(size_t size)
{
void *ret = malloc(size);
if (0 != size && 0 == ret) {
umm_last_fail_alloc_addr = __builtin_return_address(0);
umm_last_fail_alloc_size = size;
}
return ret;
}
void *operator new[](size_t size)
{
void *ret = malloc(size);
if (0 != size && 0 == ret) {
umm_last_fail_alloc_addr = __builtin_return_address(0);
umm_last_fail_alloc_size = size;
}
return ret;
}
void operator delete(void * ptr)
{
free(ptr);
}
void operator delete[](void * ptr)
{
free(ptr);
}
extern "C" void __cxa_pure_virtual(void) __attribute__ ((__noreturn__));
extern "C" void __cxa_deleted_virtual(void) __attribute__ ((__noreturn__));
@ -98,53 +68,5 @@ extern "C" void __cxa_guard_abort(__guard* pg)
xt_wsr_ps(reinterpret_cast<guard_t*>(pg)->ps);
}
namespace std
{
void __throw_bad_function_call()
{
panic();
}
void __throw_length_error(char const*)
{
panic();
}
void __throw_bad_alloc()
{
panic();
}
void __throw_logic_error(const char* str)
{
(void) str;
panic();
}
void __throw_out_of_range(const char* str)
{
(void) str;
panic();
}
void __throw_bad_cast(void)
{
panic();
}
void __throw_ios_failure(const char* str)
{
(void) str;
panic();
}
void __throw_runtime_error(const char* str)
{
(void) str;
panic();
}
} // namespace std
// TODO: rebuild windows toolchain to make this unnecessary:
void* __dso_handle;

View File

@ -134,16 +134,47 @@ static void loop_task(os_event_t *events) {
panic();
}
}
extern "C" {
struct object { long placeholder[ 10 ]; };
void __register_frame_info (const void *begin, struct object *ob);
extern char __eh_frame[];
}
static void do_global_ctors(void) {
static struct object ob;
__register_frame_info( __eh_frame, &ob );
void (**p)(void) = &__init_array_end;
while (p != &__init_array_start)
(*--p)();
}
extern "C" {
extern void __unhandled_exception(const char *str);
static void __unhandled_exception_cpp()
{
static bool terminating;
if (terminating)
abort();
terminating = true;
/* Use a trick from vterminate.cc to get any std::exception what() */
try {
__throw_exception_again;
} catch (const std::exception& e) {
__unhandled_exception( e.what() );
} catch (...) {
__unhandled_exception( "" );
}
}
}
void init_done() {
system_set_os_print(1);
gdb_init();
std::set_terminate(__unhandled_exception_cpp);
do_global_ctors();
esp_schedule();
}

View File

@ -40,6 +40,7 @@ static const char* s_panic_func = 0;
static const char* s_panic_what = 0;
static bool s_abort_called = false;
static const char* s_unhandled_exception = NULL;
void abort() __attribute__((noreturn));
static void uart_write_char_d(char c);
@ -119,6 +120,9 @@ void __wrap_system_restart_local() {
}
ets_putc('\n');
}
else if (s_unhandled_exception) {
ets_printf_P("\nUnhandled exception: %s\n", s_unhandled_exception);
}
else if (s_abort_called) {
ets_printf_P("\nAbort called\n");
}
@ -233,6 +237,11 @@ void abort() {
raise_exception();
}
void __unhandled_exception(const char *str) {
s_unhandled_exception = str;
raise_exception();
}
void __assert_func(const char *file, int line, const char *func, const char *what) {
s_panic_file = file;
s_panic_line = line;

View File

@ -128,4 +128,4 @@ void _exit(int status) {
int atexit(void (*func)()) {
(void) func;
return 0;
}
}

View File

@ -1,292 +0,0 @@
/*
pgmspace.cpp - string functions that support PROGMEM
Copyright (c) 2015 Michael C. Miller. All right reserved.
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <ctype.h>
#include <stdint.h>
#include <stddef.h>
#include <stdbool.h>
#include <stdarg.h>
#include "pgmspace.h"
extern "C" {
size_t strnlen_P(PGM_P s, size_t size) {
const char* cp;
for (cp = s; size != 0 && pgm_read_byte(cp) != '\0'; cp++, size--);
return (size_t) (cp - s);
}
char* strstr_P(const char* haystack, PGM_P needle)
{
const char* pn = reinterpret_cast<const char*>(needle);
if (haystack[0] == 0) {
if (pgm_read_byte(pn)) {
return NULL;
}
return (char*) haystack;
}
while (*haystack) {
size_t i = 0;
while (true) {
char n = pgm_read_byte(pn + i);
if (n == 0) {
return (char *) haystack;
}
if (n != haystack[i]) {
break;
}
++i;
}
++haystack;
}
return NULL;
}
void* memcpy_P(void* dest, PGM_VOID_P src, size_t count) {
const uint8_t* read = reinterpret_cast<const uint8_t*>(src);
uint8_t* write = reinterpret_cast<uint8_t*>(dest);
while (count)
{
*write++ = pgm_read_byte(read++);
count--;
}
return dest;
}
int memcmp_P(const void* buf1, PGM_VOID_P buf2P, size_t size) {
int result = 0;
const uint8_t* read1 = (const uint8_t*)buf1;
const uint8_t* read2 = (const uint8_t*)buf2P;
while (size > 0) {
uint8_t ch2 = pgm_read_byte(read2);
uint8_t ch1 = *read1;
if (ch1 != ch2) {
result = (int)(ch1)-(int)(ch2);
break;
}
read1++;
read2++;
size--;
}
return result;
}
void* memccpy_P(void* dest, PGM_VOID_P src, int c, size_t count) {
uint8_t* read = (uint8_t*)src;
uint8_t* write = (uint8_t*)dest;
void* result = NULL;
while (count > 0) {
uint8_t ch = pgm_read_byte(read++);
*write++ = ch;
count--;
if (c == ch) {
return write; // the value after the found c
}
}
return result;
}
void* memmem_P(const void* buf, size_t bufSize, PGM_VOID_P findP, size_t findPSize) {
const uint8_t* read = (const uint8_t*)buf;
const uint8_t* find = (uint8_t*)findP;
uint8_t first = pgm_read_byte(find++);
findPSize--;
while (bufSize > 0) {
if (*read == first) {
size_t findSize = findPSize;
const uint8_t* tag = read + 1;
size_t tagBufSize = bufSize - 1;
const uint8_t* findTag = find;
while (tagBufSize > 0 && findSize > 0) {
uint8_t ch = pgm_read_byte(findTag++);
if (ch != *tag) {
bufSize--;
read++;
break;
}
findSize--;
tagBufSize--;
tag++;
}
if (findSize == 0) {
return (void*)read;
}
}
else {
bufSize--;
read++;
}
}
return NULL;
}
char* strncpy_P(char* dest, PGM_P src, size_t size) {
bool size_known = (size != SIZE_IRRELEVANT);
const char* read = src;
char* write = dest;
char ch = '.';
while (size > 0 && ch != '\0')
{
ch = pgm_read_byte(read++);
*write++ = ch;
size--;
}
if (size_known)
{
while (size > 0)
{
*write++ = 0;
size--;
}
}
return dest;
}
char* strncat_P(char* dest, PGM_P src, size_t size) {
char* write = dest;
while (*write != '\0')
{
write++;
}
const char* read = src;
char ch = '.';
while (size > 0 && ch != '\0')
{
ch = pgm_read_byte(read++);
*write++ = ch;
size--;
}
if (ch != '\0')
{
*write = '\0';
}
return dest;
}
int strncmp_P(const char* str1, PGM_P str2P, size_t size) {
int result = 0;
while (size > 0)
{
char ch1 = *str1++;
char ch2 = pgm_read_byte(str2P++);
result = ch1 - ch2;
if (result != 0 || ch2 == '\0')
{
break;
}
size--;
}
return result;
}
int strncasecmp_P(const char* str1, PGM_P str2P, size_t size) {
int result = 0;
while (size > 0)
{
char ch1 = tolower(*str1++);
char ch2 = tolower(pgm_read_byte(str2P++));
result = ch1 - ch2;
if (result != 0 || ch2 == '\0')
{
break;
}
size--;
}
return result;
}
int printf_P(PGM_P formatP, ...) {
int ret;
va_list arglist;
va_start(arglist, formatP);
size_t fmtLen = strlen_P(formatP);
char* format = new char[fmtLen + 1];
strcpy_P(format, formatP);
ret = vprintf(format, arglist);
delete[] format;
va_end(arglist);
return ret;
}
int sprintf_P(char* str, PGM_P formatP, ...) {
int ret;
va_list arglist;
va_start(arglist, formatP);
ret = vsnprintf_P(str, SIZE_IRRELEVANT, formatP, arglist);
va_end(arglist);
return ret;
}
int snprintf_P(char* str, size_t strSize, PGM_P formatP, ...) {
int ret;
va_list arglist;
va_start(arglist, formatP);
ret = vsnprintf_P(str, strSize, formatP, arglist);
va_end(arglist);
return ret;
}
int vsnprintf_P(char* str, size_t strSize, PGM_P formatP, va_list ap) {
int ret;
size_t fmtLen = strlen_P(formatP);
char* format = new char[fmtLen + 1];
strcpy_P(format, formatP);
ret = vsnprintf(str, strSize, format, ap);
delete[] format;
return ret;
}
} // extern "C"

View File

@ -1,163 +1,14 @@
#ifndef __PGMSPACE_H_
#define __PGMSPACE_H_
// pgmspace.h stub
#include <stdint.h>
#include <stdio.h>
// This file's contents have been moved to newlib. This file simply
// includes the newlib pgmspace file as well as some ets headers
// to preserve backwards compatibility
#include <sys/pgmspace.h>
#ifdef __ets__
#include "ets_sys.h"
#include "osapi.h"
// Since __section__ is supposed to be only use for global variables,
// there could be conflicts when a static/inlined function has them in the
// same file as a non-static PROGMEM object.
// Ref: https://gcc.gnu.org/onlinedocs/gcc-3.2/gcc/Variable-Attributes.html
// Place each progmem object into its own named section, avoiding conflicts
// The following two macros cause a parameter to be enclosed in quotes
// by the preopressor (i.e. for concatenating ints to strings)
#define __STRINGIZE_NX(A) #A
#define __STRINGIZE(A) __STRINGIZE_NX(A)
#define PROGMEM __attribute__((section( "\".irom.text." __FILE__ "." __STRINGIZE(__LINE__) "." __STRINGIZE(__COUNTER__) "\"")))
#define PGM_P const char *
#define PGM_VOID_P const void *
#define PSTR(s) (__extension__({static const char __c[] PROGMEM = (s); &__c[0];}))
#else //__ets__
#define PROGMEM
#define PGM_P const char *
#define PGM_VOID_P const void *
#define PSTR(s) (s)
#endif // __ets__
#ifdef __cplusplus
extern "C" {
#endif
#define _SFR_BYTE(n) (n)
#ifdef __PROG_TYPES_COMPAT__
typedef void prog_void;
typedef char prog_char;
typedef unsigned char prog_uchar;
typedef int8_t prog_int8_t;
typedef uint8_t prog_uint8_t;
typedef int16_t prog_int16_t;
typedef uint16_t prog_uint16_t;
typedef int32_t prog_int32_t;
typedef uint32_t prog_uint32_t;
#endif // defined(__PROG_TYPES_COMPAT__)
#define SIZE_IRRELEVANT 0x7fffffff
// memchr_P and memrchr_P are not implemented due to danger in its use, and
// how uninteresting their use is
// since its a flash string, you should already know where the char is within it,
// further, it could return a pointer into the flash memory that is not 32bit aligned
// which could cause an exception if read
// PGM_VOID_P memchr_P(PGM_VOID_P bufP, int c, size_t count);
// PGM_VOID_P memrchr_P(PGM_VOID_P bufP, int c, size_t count);
int memcmp_P(const void* buf1, PGM_VOID_P buf2P, size_t size);
// memccpy_P is only valid when used with pointers to 8bit data, due to size aligned pointers
// and endianess of the values greater than 8bit, matching c may return invalid aligned pointers
void* memccpy_P(void* dest, PGM_VOID_P src, int c, size_t count);
void* memmem_P(const void* buf, size_t bufSize, PGM_VOID_P findP, size_t findPSize);
void* memcpy_P(void* dest, PGM_VOID_P src, size_t count);
char* strncpy_P(char* dest, PGM_P src, size_t size);
#define strcpy_P(dest, src) strncpy_P((dest), (src), SIZE_IRRELEVANT)
char* strncat_P(char* dest, PGM_P src, size_t size);
#define strcat_P(dest, src) strncat_P((dest), (src), SIZE_IRRELEVANT)
int strncmp_P(const char* str1, PGM_P str2P, size_t size);
#define strcmp_P(str1, str2P) strncmp_P((str1), (str2P), SIZE_IRRELEVANT)
int strncasecmp_P(const char* str1, PGM_P str2P, size_t size);
#define strcasecmp_P(str1, str2P) strncasecmp_P((str1), (str2P), SIZE_IRRELEVANT)
size_t strnlen_P(PGM_P s, size_t size);
#define strlen_P(strP) strnlen_P((strP), SIZE_IRRELEVANT)
char* strstr_P(const char* haystack, PGM_P needle);
int printf_P(PGM_P formatP, ...) __attribute__((format(printf, 1, 2)));
int sprintf_P(char *str, PGM_P formatP, ...) __attribute__((format(printf, 2, 3)));
int snprintf_P(char *str, size_t strSize, PGM_P formatP, ...) __attribute__((format(printf, 3, 4)));
int vsnprintf_P(char *str, size_t strSize, PGM_P formatP, va_list ap) __attribute__((format(printf, 3, 0)));
// flash memory must be read using 32 bit aligned addresses else a processor
// exception will be triggered
// order within the 32 bit values are
// --------------
// b3, b2, b1, b0
// w1, w0
#ifdef __ets__
#define pgm_read_with_offset(addr, res) \
asm("extui %0, %1, 0, 2\n" /* Extract offset within word (in bytes) */ \
"sub %1, %1, %0\n" /* Subtract offset from addr, yielding an aligned address */ \
"l32i.n %1, %1, 0x0\n" /* Load word from aligned address */ \
"slli %0, %0, 3\n" /* Mulitiply offset by 8, yielding an offset in bits */ \
"ssr %0\n" /* Prepare to shift by offset (in bits) */ \
"srl %0, %1\n" /* Shift right; now the requested byte is the first one */ \
:"=r"(res), "=r"(addr) \
:"1"(addr) \
:);
static inline uint8_t pgm_read_byte_inlined(const void* addr) {
register uint32_t res;
pgm_read_with_offset(addr, res);
return (uint8_t) res; /* This masks the lower byte from the returned word */
}
/* Although this says "word", it's actually 16 bit, i.e. half word on Xtensa */
static inline uint16_t pgm_read_word_inlined(const void* addr) {
register uint32_t res;
pgm_read_with_offset(addr, res);
return (uint16_t) res; /* This masks the lower half-word from the returned word */
}
// Make sure, that libraries checking existence of this macro are not failing
#ifdef __PROG_TYPES_COMPAT__
#define pgm_read_byte(addr) pgm_read_byte_inlined((const void*)(addr))
#define pgm_read_word(addr) pgm_read_word_inlined((const void*)(addr))
#else
#define pgm_read_byte(addr) pgm_read_byte_inlined(addr)
#define pgm_read_word(addr) pgm_read_word_inlined(addr)
#endif
#else //__ets__
#define pgm_read_byte(addr) (*reinterpret_cast<const uint8_t*>(addr))
#define pgm_read_word(addr) (*reinterpret_cast<const uint16_t*>(addr))
#endif //__ets__
#define pgm_read_dword(addr) (*reinterpret_cast<const uint32_t*>(addr))
#define pgm_read_float(addr) (*reinterpret_cast<const float*>(addr))
#define pgm_read_ptr(addr) (*reinterpret_cast<const void* const *>(addr))
#define pgm_read_byte_near(addr) pgm_read_byte(addr)
#define pgm_read_word_near(addr) pgm_read_word(addr)
#define pgm_read_dword_near(addr) pgm_read_dword(addr)
#define pgm_read_float_near(addr) pgm_read_float(addr)
#define pgm_read_ptr_near(addr) pgm_read_ptr(addr)
#define pgm_read_byte_far(addr) pgm_read_byte(addr)
#define pgm_read_word_far(addr) pgm_read_word(addr)
#define pgm_read_dword_far(addr) pgm_read_dword(addr)
#define pgm_read_float_far(addr) pgm_read_float(addr)
#define pgm_read_ptr_far(addr) pgm_read_ptr(addr)
#ifdef __cplusplus
}
#endif
#endif //__PGMSPACE_H_