1
0
mirror of https://github.com/esp8266/Arduino.git synced 2025-07-01 03:26:58 +03:00
Files
esp8266/tests/host/Makefile
Earle F. Philhower, III 6280e98b03 Enable exceptions, update to optimized newlib, migrate to new toolchain (#5376)
* Move to PROGMEM aware libc, allow PSTR in printf()

A Newlib (libc) patch is in progress to move the _P functions from inside
Arduino into first-class citizens in libc.  This Arduino patch cleans up
code that's been migrated there.  Binaries for the new libs are included
because it seems they're part of the Arduino git tree, and should be
replaced with @igrr built ones when/if the Newlib changes are accepted.

Notable changes/additions for Arduino:
Allow for use of PROGMEM based format and parameter strings in all
*printf functions.  No need for copying PSTR()s into RAM before printing
them out (transparently saves heap space when using _P functions) and
makes it easier to print out constant strings for applications.

Add "%S" (capital-S) format that I've been told, but cannot verify,
is used in Arduino to specify a PROGMEM string parameter in printfs,
as an alias for "%s" since plain "%s" can now handle PROGMEM.

Optimized the memcpy_P, strnlen_P, and strncpy_P functions to use 32-bit
direct reads whenver possible (source and dest alignment mediated), but
there is still room for improvement in others.

Finally, move several constant arrays from RODATA into PROGMEM and
update their accessors.  Among these are the ctype array, ~260 bytes,
mprec* arrays, ~300 bytes, and strings/daycounts in the time
formatting functions, ~200 bytes.  All told, sketches will see from
300 to 800 additional RAM heap free on startup (depending on their
use of these routines).

* Fix merge error in #ifdef/#endif

* Fix host test using the newlib generic pgmspace.h

Host tests now use the sys/pgmspace.h for compiles instead of the
ESP8266-specific version.

* Update with rebuilt libraries using latest newlib

* Include binaries built directly from @igrr repo

Rebuild the binaries using a git clone of
https://github.com/igrr/newlib-xtensa

Build commands for posterity:
````
rm -rf ./xtensa-lx106-elf/
./configure --prefix=<DIR>/esp8266/tools/sdk/libc --with-newlib \
            --enable-multilib --disable-newlib-io-c99-formats \
            --disable-newlib-supplied-syscalls \
            --enable-newlib-nano-formatted-io --enable-newlib-reent-small \
            --enable-target-optspace \
            --program-transform-name="s&^&xtensa-lx106-elf-&" \
            --disable-option-checking --with-target-subdir=xtensa-lx106-elf \
            --target=xtensa-lx106-elf
rm -f etc/config.cache
CROSS_CFLAGS="-fno-omit-frame-pointer -DSIGNAL_PROVIDED -DABORT_PROVIDED"\
             " -DMALLOC_PROVIDED" \
  PATH=<DIR>/esp8266/tools/xtensa-lx106-elf/bin/:$PATH \
  make all install
````

* Fix merge define conflict in c_types.h

* Fix strlen_P misaligned source error

Include fix from newlib-xtensa/fix-strlen branch cleaning up misaligned
access on a non-aligned source string.

* Fix strlen_P and strcpy_P edge cases

Ran the included test suite on ESP8266 tstring.c with the following defines:
 #define MAX_1 50
 #define memcmp memcmp_P
 #define memcpy memcpy_P
 #define memmem memmem_P
 #define memchr memchr_P
 #define strcat strcat_P
 #define strncat strncat_P
 #define strcpy strcpy_P
 #define strlen strlen_P
 #define strnlen strnlen_P
 #define strcmp strcmp_P
 #define strncmp strncmp_P

Uncovered edge case and return value problems in the optimized versions of
the strnlen_P and strncpy_P functions.  Corrected.

* Fix memcpy_P return value

memcpy-1.c test suite showed error in return value of memcpy_P.  Correct it.

* Fix strnlen_P/strlen_P off-by-4 error

Random crashes, often on String constructors using a PSTR, would occur due
to the accelerated strnlen_P going past the end of the string. Would make
debug builds fail, too (ESP.getVersionString() failure).

Fix to fall through to normal copy on a word that's got a 0 byte anywhere
in it.

* Add device tests for libc functional verification

Add test suite used to debug libc optimized _P functions to the device
tests.

* Rebuild from igrr's repo (same source as prior)

Rebuild .a from igrr's repo at 347260af117b4177389e69fd4d04169b11d87a97

* WIP - add exceptions

* Fix exception to have 0-terminator

* Move some exception constants to TEXT from RODATA

* Remove throw stubs

* Move more exception stuff to ROM

* Enable exceptions in platform.io

* Remove atexit, is duplicated in rebuilt lib

Need to look at the quick-toolchain options, there seems to be a definition
for atexit defined there (libgcc?) that needs to be excised.  For now,
remove our local do-nothing copy.

* Update libgcc to remove soft-fp functions

The esp-quick-toolchain generated libgcc.a needed to have the soft-FP routines
that are in ROM removed from it.  Remove them in the new esp-quick-toolchain
and update.

* Fix merge typos in Makefile

* Add unhandled exception handler to postmortem

* Return our atexit() handler

* Latest stdc++, minimize exception emercengy area

* Remove atexit from newlib

atexit was defined in newlib strongly, but we also define a noop atexit in core.
Since we never exit, use the core's noop and delete the atexit from libc.a

Updated in esp-quick-toolchain as well.

* Move __FUNCTION__ static strings to PROGMEM

__FUNCTION__ is unlikely to be a timing sensitive variable, so move it to
PROGMEM and not RODATA (RAM) using linker magic.

asserts() now should take no RAM for any strings.

* Clean up linker file, update to latest stdc++

* Update to latest stdc++ which doesn't call strerror

* Update to GCC5.1 exception emergency allocator

Using GCC 5.1's emergency memory allocator for exceptions, much less
space is required in programs which do not use exceptions and when
space is allocated it is managed more efficiently.

* Initial try with new compiler toolchain

* Include newlib built from esp-quick-toolchain

* Update JSON with all new esp-quick-toolchain builds

* Use 64bit Windows compiler on 64bit Windows

* Dump std::exception.what() when possible

When doing the panic on unhandled exceptions, try and grab the
.what() pointer and dump it as part of the termination info.
Makes it easy to see mem errors (std::bad_alloc) or std::runtime_error
strings.

* Use scripted install from esp-quick-toolchain

Makes sure proper libraries and includes are present by using a
scripted installation from esp-quick-install instead of a manual
one.

* Update eqk to remove atexit, fix packaging diff
2018-12-03 03:37:14 -03:00

334 lines
8.7 KiB
Makefile

BINDIR := bin
LCOV_DIRECTORY := lcov
OUTPUT_BINARY := $(BINDIR)/host_tests
CORE_PATH := ../../cores/esp8266
FORCE32 ?= 1
OPTZ ?= -Os
V ?= 0
MAKEFILE = $(word 1, $(MAKEFILE_LIST))
# I wasn't able to build with clang when -coverage flag is enabled, forcing GCC on OS X
ifeq ($(shell uname -s),Darwin)
CC ?= gcc
CXX ?= g++
endif
GCOV ?= gcov
VALGRIND ?= valgrind
LCOV ?= lcov
GENHTML ?= genhtml
ifeq ($(FORCE32),1)
ABILITY32 = $(shell echo 'int main(){return sizeof(long);}'|$(CXX) -m32 -x c++ - -o sizeoflong 2>/dev/null && ./sizeoflong; echo $$?; rm -f sizeoflong;)
ifneq ($(ABILITY32),4)
$(warning Cannot compile in 32 bit mode, switching to native mode)
else
N32 = 32
M32 = -m32
endif
endif
ifeq ($(N32),32)
$(warning compiling in 32 bits mode)
else
$(warning compiling in native mode)
endif
ifeq ($(V), 0)
VERBC = @echo "C $@";
VERBCXX = @echo "C++ $@";
VERBLD = @echo "LD $@";
VERBAR = @echo "AR $@";
else
VERBC =
VERBCXX =
VERBLD =
VERBAR =
endif
$(shell mkdir -p $(BINDIR))
CORE_CPP_FILES := $(addprefix $(CORE_PATH)/,\
StreamString.cpp \
Stream.cpp \
WString.cpp \
Print.cpp \
FS.cpp \
spiffs_api.cpp \
MD5Builder.cpp \
)
CORE_C_FILES := $(addprefix $(CORE_PATH)/,\
core_esp8266_noniso.c \
spiffs/spiffs_cache.c \
spiffs/spiffs_check.c \
spiffs/spiffs_gc.c \
spiffs/spiffs_hydrogen.c \
spiffs/spiffs_nucleus.c \
libb64/cencode.c \
)
MOCK_CPP_FILES_COMMON := $(addprefix common/,\
Arduino.cpp \
spiffs_mock.cpp \
WMath.cpp \
MockSerial.cpp \
MockTools.cpp \
)
MOCK_CPP_FILES := $(MOCK_CPP_FILES_COMMON) $(addprefix common/,\
ArduinoCatch.cpp \
)
MOCK_CPP_FILES_EMU := $(MOCK_CPP_FILES_COMMON) $(addprefix common/,\
ArduinoMain.cpp \
user_interface.cpp \
)
MOCK_C_FILES := $(addprefix common/,\
md5.c \
noniso.c \
)
INC_PATHS += $(addprefix -I, \
. \
common \
$(CORE_PATH) \
)
INC_PATHS += $(addprefix -I,\
$(shell echo ../../libraries/*/src) \
$(shell echo ../../libraries/*) \
../../tools/sdk/include \
../../tools/sdk/lwip2/include \
)
TEST_CPP_FILES := \
fs/test_fs.cpp \
core/test_pgmspace.cpp \
core/test_md5builder.cpp \
core/test_string.cpp \
core/test_PolledTimeout.cpp
PREINCLUDES := \
-include common/mock.h \
-include common/c_types.h \
ifneq ($(D),)
OPTZ=-O0
DEBUG += -DDEBUG_ESP_PORT=Serial
DEBUG += -DDEBUG_ESP_SSL -DDEBUG_ESP_TLS_MEM -DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_HTTP_SERVER -DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_UPDATER -DDEBUG_ESP_OTA -DDEBUG_ESP_MDNS
endif
FLAGS += $(DEBUG) -Wall -coverage $(OPTZ) -fno-common -g $(M32)
FLAGS += -DHTTPCLIENT_1_1_COMPATIBLE=0
FLAGS += -DLWIP_IPV6=0
FLAGS += $(MKFLAGS)
CXXFLAGS += -std=c++11 $(FLAGS)
CFLAGS += -std=c99 $(FLAGS)
LDFLAGS += -coverage $(OPTZ) -g $(M32)
VALGRINDFLAGS += --leak-check=full --track-origins=yes --error-limit=no --show-leak-kinds=all --error-exitcode=999
CXXFLAGS += -Wno-error=format-security # cores/esp8266/Print.cpp:42:24: error: format not a string literal and no format arguments [-Werror=format-security] -- (os_printf_plus(not_the_best_way))
#CXXFLAGS += -Wno-format-security # cores/esp8266/Print.cpp:42:40: warning: format not a string literal and no format arguments [-Wformat-security] -- (os_printf_plus(not_the_best_way))
remduplicates = $(strip $(if $1,$(firstword $1) $(call remduplicates,$(filter-out $(firstword $1),$1))))
C_SOURCE_FILES = $(MOCK_C_FILES) $(CORE_C_FILES)
CPP_SOURCE_FILES = $(MOCK_CPP_FILES) $(CORE_CPP_FILES) $(TEST_CPP_FILES)
C_OBJECTS = $(C_SOURCE_FILES:.c=.c.o)
CPP_OBJECTS_CORE = $(MOCK_CPP_FILES:.cpp=.cpp.o) $(CORE_CPP_FILES:.cpp=.cpp.o)
CPP_OBJECTS_TESTS = $(TEST_CPP_FILES:.cpp=.cpp.o)
CPP_OBJECTS = $(CPP_OBJECTS_CORE) $(CPP_OBJECTS_TESTS)
OBJECTS = $(C_OBJECTS) $(CPP_OBJECTS)
COVERAGE_FILES = $(OBJECTS:.o=.gc*)
all: help
CI:
$(MAKE) -f $(MAKEFILE) MKFLAGS=-Werror FORCE32=0 OPTZ=-O0 doCI
doCI: build-info $(OUTPUT_BINARY) valgrind test gcov # run CI
test: $(OUTPUT_BINARY) # run host test for CI
$(OUTPUT_BINARY)
clean: clean-objects clean-coverage # clean everything
rm -rf $(BINDIR)
clean-objects:
rm -rf $(C_OBJECTS) $(CPP_OBJECTS_CORE) $(CPP_OBJECTS_CORE_EMU) $(CPP_OBJECTS_TESTS)
clean-coverage:
rm -rf $(COVERAGE_FILES) $(LCOV_DIRECTORY) *.gcov
gcov: test # run coverage for CI
find $(CORE_PATH) -name "*.gcno" -exec $(GCOV) -r -pb {} +
valgrind: $(OUTPUT_BINARY)
mkdir -p $(LCOV_DIRECTORY)
$(LCOV) --directory ../../cores/esp8266/ --zerocounters
$(VALGRIND) $(VALGRINDFLAGS) $(OUTPUT_BINARY)
$(LCOV) --directory $(CORE_PATH) --capture --output-file $(LCOV_DIRECTORY)/app.info
-$(GENHTML) $(LCOV_DIRECTORY)/app.info -o $(LCOV_DIRECTORY)
build-info: # show toolchain version
@echo "-------- build tools info --------"
@echo "CC: " $(CC)
$(CC) -v
@echo "CXX: " $(CXX)
$(CXX) -v
@echo "GCOV: " $(GCOV)
$(GCOV) -v
@echo "----------------------------------"
-include $(BINDIR)/.*.d
.SUFFIXES:
%.c.o: %.c
$(VERBC) $(CC) $(PREINCLUDES) $(CFLAGS) $(INC_PATHS) -MD -MF $(BINDIR)/.$(notdir $<).d -c -o $@ $<
.PRECIOUS: %.cpp.o
%.cpp.o: %.cpp
$(VERBCXX) $(CXX) $(PREINCLUDES) $(CXXFLAGS) $(INC_PATHS) -MD -MF $(BINDIR)/.$(notdir $<).d -c -o $@ $<
$(BINDIR)/core.a: $(C_OBJECTS) $(CPP_OBJECTS_CORE)
ar -rcu $@ $^
ranlib -c $@
$(OUTPUT_BINARY): $(CPP_OBJECTS_TESTS) $(BINDIR)/core.a
$(VERBLD) $(CXX) $(LDFLAGS) $^ -o $@
#################################################
# building ino sources
ARDUINO_LIBS := \
$(addprefix $(CORE_PATH)/,\
IPAddress.cpp \
Updater.cpp \
base64.cpp \
) \
$(addprefix ../../libraries/,\
$(addprefix ESP8266WiFi/src/,\
ESP8266WiFi.cpp \
ESP8266WiFiAP.cpp \
ESP8266WiFiGeneric.cpp \
ESP8266WiFiMulti.cpp \
ESP8266WiFiSTA-WPS.cpp \
ESP8266WiFiSTA.cpp \
ESP8266WiFiScan.cpp \
WiFiClient.cpp \
WiFiUdp.cpp \
WiFiClientSecureBearSSL.cpp \
WiFiServerSecureBearSSL.cpp \
BearSSLHelpers.cpp \
CertStoreBearSSL.cpp \
) \
$(addprefix ESP8266WebServer/src/,\
ESP8266WebServer.cpp \
Parsing.cpp \
detail/mimetable.cpp \
) \
ESP8266mDNS/ESP8266mDNS.cpp \
ArduinoOTA/ArduinoOTA.cpp \
DNSServer/src/DNSServer.cpp \
ESP8266AVRISP/src/ESP8266AVRISP.cpp \
ESP8266HTTPClient/src/ESP8266HTTPClient.cpp \
) \
MOCK_ARDUINO_LIBS := \
common/ClientContextSocket.cpp \
common/ClientContextTools.cpp \
common/MockWiFiServerSocket.cpp \
common/MockWiFiServer.cpp \
common/UdpContextSocket.cpp \
common/HostWiring.cpp \
common/MockEsp.cpp \
common/MockEEPROM.cpp \
common/MockSPI.cpp \
CPP_SOURCES_CORE_EMU = \
$(MOCK_CPP_FILES_EMU) \
$(CORE_CPP_FILES) \
$(MOCK_ARDUINO_LIBS) \
$(ARDUINO_LIBS) \
LIBSSLFILE = ../../tools/sdk/ssl/bearssl/build$(N32)/libbearssl.a
ifeq (,$(wildcard $(LIBSSLFILE)))
LIBSSL =
else
LIBSSL = $(LIBSSLFILE)
endif
ssl: # download source and build BearSSL
cd ../../tools/sdk/ssl && make native$(N32)
ULIBPATHS = $(shell echo $(ULIBDIRS) | sed 's,:, ,g')
USERLIBDIRS = $(shell test -z "$(ULIBPATHS)" || for d in $(ULIBPATHS); do for dd in $$d $$d/src; do test -d $$dd && { echo -I$$dd; echo "userlib: using directory '$$dd'" 1>&2; } done; done)
USERLIBSRCS = $(shell test -z "$(ULIBPATHS)" || for d in $(ULIBPATHS); do for ss in $$d/*.cpp $$d/src/*.cpp; do test -r $$ss && echo $$ss; done; done)
INC_PATHS += $(USERLIBDIRS)
INC_PATHS += -I$(INODIR)/..
CPP_OBJECTS_CORE_EMU = $(CPP_SOURCES_CORE_EMU:.cpp=.cpp.o) $(USERLIBSRCS:.cpp=.cpp.o)
bin/fullcore.a: $(C_OBJECTS) $(CPP_OBJECTS_CORE_EMU)
$(VERBAR) ar -rcu $@ $^
$(VERBAR) ranlib -c $@
%: %.ino.cpp.o bin/fullcore.a
$(VERBLD) $(CXX) $(LDFLAGS) $< bin/fullcore.a $(LIBSSL) -o $@
@echo "----> $@ <----"
#################################################
# are we in primary make call ?
ifeq ($(INO),)
%: %.ino
@# recursive 'make' with paths
$(MAKE) -f $(MAKEFILE) MKFLAGS=-Wextra INODIR=$(dir $@) INO=$(notdir $@) $(BINDIR)/$(notdir $@)/$(notdir $@)
@# see below the new build rule with fixed output path outside from core location
#####################
# recursive call on ino target
else
$(BINDIR)/$(INO)/$(INO).ino.cpp:
@# arduino builder would come around here (.ino -> .ino.cpp)
@mkdir -p $(BINDIR)/$(INO); \
( \
echo "#include <$(INODIR)/$(INO).ino>"; \
for i in $(INODIR)/*.ino; do \
test "$$i" = $(INODIR)/$(INO).ino || echo "#include \"$$i\""; \
done; \
) > $(BINDIR)/$(INO)/$(INO).ino.cpp
endif # recursive
#####################
#################################################
.PHONY: list
list: # show core example list
@for dir in ../../libraries/*/examples; do \
exampledir=$${dir%/*}; \
exampledirname=$${exampledir##*/}; \
for subdir in $$dir/*; do \
exname=$${subdir##*/}; \
echo "$$subdir/$$exname"; \
done; \
done; \
#################################################
# help
.PHONY: help
help:
@cat README.txt
@echo ""
@echo "Make rules:"
@echo ""
@sed -rne 's,([^: \t]*):[^=#]*#[\t ]*(.*),\1 - \2,p' $(MAKEFILE)
@echo ""