1
0
mirror of https://github.com/esp8266/Arduino.git synced 2025-10-28 17:15:26 +03:00
Files
esp8266/tests/host/Makefile
Earle F. Philhower, III b1da9eda46 SD Filesystem compatible with 8266 File, using latest SdFat (#5525)
* Add a FAT filesystem for SD cards to Arduino FS

Arduino forked a copy of SD lib several years ago, put their own wrapper
around it, and it's been languishing in our ESP8266 libraries ever since
as SD. It doesn't support long file names, has class names which
conflict with the ESP8266 internal names, and hasn't been updated in
ages.

The original author of the SD library has continued work in the
meantime, and produced a very feature rich implementation of SdFat. It
unfortunately also conflicts with the class names we use in ESP8266
Arduino and has a different API than the internal SPIFFS or proposed
LittleFS filesystem objects.

This PR puts a wrapper around the latest and greatest SdFat library,
by forking it and wrapping its classes in a private namespace "sdfat,"
and making as thin a wrapper as possible around it to conform to
the ESP8266 FS, File, and Dir classes.

This PR also removes the Arduino SD.h class library and rewrites it
using the new SDFS filesystem to make everything in the ESP8266
Arduino core compatible with each other.

By doing so it lets us use a single interface for anything needing a
file instead of multiple ones (see SDWebServer and how a different
object is needed vs. one serving from SPIFFS even though the logic is
all the same). Same for BearSSL's CertStores and probably a few others
I've missed, cleaning up our code base significantly.

Like LittleFS, silently create directories when a file is created with
a subdirectory specifier ("/path/to/file.txt") if they do not yet exist.

Adds a blacklist of sketches to skip in the CI process (because SdFat
has many examples which do not build properly on the ESP8266).

Now that LittleFS and SDFS have directory support, the FS needs to be
able to communicate whether a name is one or the other.  Add a simple
bool FS::isDirectory() and bool FS::isFile() method.  SPIFFS doesn't
have directories, so if it's valid it's a file and reported as such.

Add ::mkdir/::rmdir to the FS class to allow users to make and destroy
subdirectories.  SPIFFS directory operations will, of course, fail
and return false.

Emulate a 16MB SD card and allow test runner to exercise it by using
a custom SdFat HOST_MOCK-enabled object.

Throw out the original Arduino SD.h class and rewrite from scratch using
only the ESP8266 native SDFS calls.  This makes "SD" based applications
compatible with normal ESP8266 "File" and "FS" and "SPIFFS" operations.

The only major visible change for users is that long filenames now are
fully supported and work without any code changes.  If there are static
arrays of 11 bytes for old 8.3 names in code, they will need to be
adjusted.

While it is recommended to use the more powerful SDFS class to access SD
cards, this SD.h wrapper allows for use of existing Arduino libraries
which are built to only with with that SD class.

Additional helper functions added to ESP8266 native Filesystem:: classes
to help support this portability.

The rewrite is good enough to run the original SDWebServer and SD
example code without any changes.

* Add a FSConfig and SDFSConfig param to FS.begin()

Allows for configuration values to be passed into a filesystem via the
begin method.  By default, a FS will receive a nullptr and should so
whatever is appropriate.

The base FSConfig class has one parameter, _autoFormat, set by the
default constructor to true.

For SPIFFS, you can now disable auto formatting on mount failure by
passing in a FSConfig(false) object.

For SDFS a SDFSConfig parameter can be passed into config specifying the
chip select and SPI configuration.  If nothing is passed in, the begin
will fail since there are no safe default values here.

* Add FS::setConfig to set FS-specific options

Add a new call, FS::setConfig(const {SDFS,SPIFFS}Config *cfg), which
takes a FS-specific configuration object and copies any special settings
on a per-FS basis.  The call is only valid on unmounted filesystems, and
checks the type of object passed in matches the FS being configured.

Updates the docs and tests to utilize this new configuration method.

* Add ::truncate to File interface

Fixes #3846

* Use polledTimeout for formatting yields, cleanup

Use the new polledTimeout class to ensure a yield every 5ms while
formatting.

Add in default case handling and some debug messages when invalid inputs
specified.

* Make setConfig take const& ref, cleaner code

setConfig now can take a parameter defined directly in the call by using
a const &ref to it, leading to one less line of code to write and
cleaner reading of the code.

Also clean up SDFS implementation pointer definition.
2019-03-06 02:14:44 +00:00

363 lines
9.4 KiB
Makefile

BINDIR := bin
LCOV_DIRECTORY := lcov
OUTPUT_BINARY := $(BINDIR)/host_tests
CORE_PATH := ../../cores/esp8266
LIBRARIES_PATH := ../../libraries
FORCE32 ?= 1
OPTZ ?= -Os
V ?= 0
MAKEFILE = $(word 1, $(MAKEFILE_LIST))
# I wasn't able to build with clang when -coverage flag is enabled, forcing GCC on OS X
ifeq ($(shell uname -s),Darwin)
CC ?= gcc
CXX ?= g++
endif
GCOV ?= gcov
VALGRIND ?= valgrind
LCOV ?= lcov
GENHTML ?= genhtml
ifeq ($(FORCE32),1)
SIZEOFLONG = $(shell echo 'int main(){return sizeof(long);}'|$(CXX) -m32 -x c++ - -o sizeoflong 2>/dev/null && ./sizeoflong; echo $$?; rm -f sizeoflong;)
ifneq ($(SIZEOFLONG),4)
$(warning Cannot compile in 32 bit mode, switching to native mode)
else
N32 = 32
M32 = -m32
endif
endif
ifeq ($(N32),32)
$(warning compiling in 32 bits mode)
else
$(warning compiling in native mode)
endif
ifeq ($(V), 0)
VERBC = @echo "C $@";
VERBCXX = @echo "C++ $@";
VERBLD = @echo "LD $@";
VERBAR = @echo "AR $@";
else
VERBC =
VERBCXX =
VERBLD =
VERBAR =
endif
$(shell mkdir -p $(BINDIR))
CORE_CPP_FILES := $(addprefix $(CORE_PATH)/,\
StreamString.cpp \
Stream.cpp \
WString.cpp \
Print.cpp \
FS.cpp \
spiffs_api.cpp \
MD5Builder.cpp \
core_esp8266_noniso.cpp \
spiffs/spiffs_cache.cpp \
spiffs/spiffs_check.cpp \
spiffs/spiffs_gc.cpp \
spiffs/spiffs_hydrogen.cpp \
spiffs/spiffs_nucleus.cpp \
libb64/cencode.cpp \
libb64/cdecode.cpp \
Schedule.cpp \
HardwareSerial.cpp \
) \
$(addprefix $(LIBRARIES_PATH)/ESP8266SdFat/src/, \
FatLib/FatFile.cpp \
FatLib/FatFileLFN.cpp \
FatLib/FatFilePrint.cpp \
FatLib/FatFileSFN.cpp \
FatLib/FatVolume.cpp \
FatLib/FmtNumber.cpp \
FatLib/StdioStream.cpp \
) \
$(LIBRARIES_PATH)/SDFS/src/SDFS.cpp \
$(LIBRARIES_PATH)/SD/src/SD.cpp
CORE_C_FILES := $(addprefix $(CORE_PATH)/,\
)
MOCK_CPP_FILES_COMMON := $(addprefix common/,\
Arduino.cpp \
spiffs_mock.cpp \
sdfs_mock.cpp \
WMath.cpp \
MockUART.cpp \
MockTools.cpp \
MocklwIP.cpp \
)
MOCK_CPP_FILES := $(MOCK_CPP_FILES_COMMON) $(addprefix common/,\
ArduinoCatch.cpp \
)
MOCK_CPP_FILES_EMU := $(MOCK_CPP_FILES_COMMON) $(addprefix common/,\
ArduinoMain.cpp \
ArduinoMainUdp.cpp \
ArduinoMainSpiffs.cpp \
user_interface.cpp \
)
MOCK_C_FILES := $(addprefix common/,\
md5.c \
noniso.c \
)
INC_PATHS += $(addprefix -I, \
. \
common \
$(CORE_PATH) \
)
INC_PATHS += $(addprefix -I,\
$(shell echo ../../libraries/*/src) \
$(shell echo ../../libraries/*) \
../../tools/sdk/include \
../../tools/sdk/lwip2/include \
)
TEST_CPP_FILES := \
fs/test_fs.cpp \
core/test_pgmspace.cpp \
core/test_md5builder.cpp \
core/test_string.cpp \
core/test_PolledTimeout.cpp
PREINCLUDES := \
-include common/mock.h \
-include common/c_types.h \
ifneq ($(D),)
OPTZ=-O0
DEBUG += -DDEBUG_ESP_PORT=Serial
DEBUG += -DDEBUG_ESP_SSL -DDEBUG_ESP_TLS_MEM -DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_HTTP_SERVER -DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_UPDATER -DDEBUG_ESP_OTA -DDEBUG_ESP_MDNS
endif
FLAGS += $(DEBUG) -Wall -coverage $(OPTZ) -fno-common -g $(M32)
FLAGS += -DHTTPCLIENT_1_1_COMPATIBLE=0
FLAGS += -DLWIP_IPV6=0
FLAGS += -DHOST_MOCK=1
FLAGS += -DNONOSDK221=1
FLAGS += $(MKFLAGS)
CXXFLAGS += -std=c++11 $(FLAGS)
CFLAGS += -std=c99 $(FLAGS)
LDFLAGS += -coverage $(OPTZ) -g $(M32)
VALGRINDFLAGS += --leak-check=full --track-origins=yes --error-limit=no --show-leak-kinds=all --error-exitcode=999
CXXFLAGS += -Wno-error=format-security # cores/esp8266/Print.cpp:42:24: error: format not a string literal and no format arguments [-Werror=format-security] -- (os_printf_plus(not_the_best_way))
#CXXFLAGS += -Wno-format-security # cores/esp8266/Print.cpp:42:40: warning: format not a string literal and no format arguments [-Wformat-security] -- (os_printf_plus(not_the_best_way))
remduplicates = $(strip $(if $1,$(firstword $1) $(call remduplicates,$(filter-out $(firstword $1),$1))))
C_SOURCE_FILES = $(MOCK_C_FILES) $(CORE_C_FILES)
CPP_SOURCE_FILES = $(MOCK_CPP_FILES) $(CORE_CPP_FILES) $(TEST_CPP_FILES)
C_OBJECTS = $(C_SOURCE_FILES:.c=.c.o)
CPP_OBJECTS_CORE = $(MOCK_CPP_FILES:.cpp=.cpp.o) $(CORE_CPP_FILES:.cpp=.cpp.o)
CPP_OBJECTS_TESTS = $(TEST_CPP_FILES:.cpp=.cpp.o)
CPP_OBJECTS = $(CPP_OBJECTS_CORE) $(CPP_OBJECTS_TESTS)
OBJECTS = $(C_OBJECTS) $(CPP_OBJECTS)
COVERAGE_FILES = $(OBJECTS:.o=.gc*)
all: help
CI: # run CI
$(MAKE) -f $(MAKEFILE) MKFLAGS=-Werror FORCE32=0 OPTZ=-O0 doCI
doCI: build-info $(OUTPUT_BINARY) valgrind test gcov
test: $(OUTPUT_BINARY) # run host test for CI
$(OUTPUT_BINARY)
clean: clean-objects clean-coverage # clean everything
rm -rf $(BINDIR)
clean-objects:
rm -rf $(C_OBJECTS) $(CPP_OBJECTS_CORE) $(CPP_OBJECTS_CORE_EMU) $(CPP_OBJECTS_TESTS)
clean-coverage:
rm -rf $(COVERAGE_FILES) $(LCOV_DIRECTORY) *.gcov
gcov: test # run coverage for CI
find $(CORE_PATH) -name "*.gcno" -exec $(GCOV) -r -pb {} +
valgrind: $(OUTPUT_BINARY)
mkdir -p $(LCOV_DIRECTORY)
$(LCOV) --directory ../../cores/esp8266/ --zerocounters
$(VALGRIND) $(VALGRINDFLAGS) $(OUTPUT_BINARY)
$(LCOV) --directory $(CORE_PATH) --capture --output-file $(LCOV_DIRECTORY)/app.info
-$(GENHTML) $(LCOV_DIRECTORY)/app.info -o $(LCOV_DIRECTORY)
build-info: # show toolchain version
@echo "-------- build tools info --------"
@echo "CC: " $(CC)
$(CC) -v
@echo "CXX: " $(CXX)
$(CXX) -v
@echo "GCOV: " $(GCOV)
$(GCOV) -v
@echo "----------------------------------"
-include $(BINDIR)/.*.d
.SUFFIXES:
%.c.o: %.c
$(VERBC) $(CC) $(PREINCLUDES) $(CFLAGS) $(INC_PATHS) -MD -MF $(BINDIR)/.$(notdir $<).d -c -o $@ $<
.PRECIOUS: %.cpp.o
%.cpp.o: %.cpp
$(VERBCXX) $(CXX) $(PREINCLUDES) $(CXXFLAGS) $(INC_PATHS) -MD -MF $(BINDIR)/.$(notdir $<).d -c -o $@ $<
$(BINDIR)/core.a: $(C_OBJECTS) $(CPP_OBJECTS_CORE)
ar -rcu $@ $^
ranlib -c $@
$(OUTPUT_BINARY): $(CPP_OBJECTS_TESTS) $(BINDIR)/core.a
$(VERBLD) $(CXX) $(LDFLAGS) $^ -o $@
#################################################
# building ino sources
ARDUINO_LIBS := \
$(addprefix $(CORE_PATH)/,\
IPAddress.cpp \
Updater.cpp \
base64.cpp \
) \
$(addprefix ../../libraries/ESP8266WiFi/src/,\
ESP8266WiFi.cpp \
ESP8266WiFiAP.cpp \
ESP8266WiFiGeneric.cpp \
ESP8266WiFiMulti.cpp \
ESP8266WiFiSTA-WPS.cpp \
ESP8266WiFiSTA.cpp \
ESP8266WiFiScan.cpp \
WiFiClient.cpp \
WiFiUdp.cpp \
WiFiClientSecureBearSSL.cpp \
WiFiServerSecureBearSSL.cpp \
BearSSLHelpers.cpp \
CertStoreBearSSL.cpp \
)
OPT_ARDUINO_LIBS ?= $(addprefix ../../libraries/,\
$(addprefix ESP8266WebServer/src/,\
ESP8266WebServer.cpp \
Parsing.cpp \
detail/mimetable.cpp \
) \
$(addprefix ESP8266mDNS/src/,\
LEAmDNS.cpp \
LEAmDNS_Control.cpp \
LEAmDNS_Helpers.cpp \
LEAmDNS_Structs.cpp \
LEAmDNS_Transfer.cpp \
ESP8266mDNS.cpp \
) \
ArduinoOTA/ArduinoOTA.cpp \
DNSServer/src/DNSServer.cpp \
ESP8266AVRISP/src/ESP8266AVRISP.cpp \
ESP8266HTTPClient/src/ESP8266HTTPClient.cpp \
)
MOCK_ARDUINO_LIBS := $(addprefix common/,\
ClientContextSocket.cpp \
ClientContextTools.cpp \
MockWiFiServerSocket.cpp \
MockWiFiServer.cpp \
UdpContextSocket.cpp \
HostWiring.cpp \
MockEsp.cpp \
MockEEPROM.cpp \
MockSPI.cpp \
)
CPP_SOURCES_CORE_EMU = \
$(MOCK_CPP_FILES_EMU) \
$(CORE_CPP_FILES) \
$(MOCK_ARDUINO_LIBS) \
$(OPT_ARDUINO_LIBS) \
$(ARDUINO_LIBS) \
LIBSSLFILE = ../../tools/sdk/ssl/bearssl/build$(N32)/libbearssl.a
ifeq (,$(wildcard $(LIBSSLFILE)))
LIBSSL =
else
LIBSSL = $(LIBSSLFILE)
endif
ssl: # download source and build BearSSL
cd ../../tools/sdk/ssl && make native$(N32)
ULIBPATHS = $(shell echo $(ULIBDIRS) | sed 's,:, ,g')
USERLIBDIRS = $(shell test -z "$(ULIBPATHS)" || for d in $(ULIBPATHS); do for dd in $$d $$d/src; do test -d $$dd && { echo -I$$dd; echo "userlib: using directory '$$dd'" 1>&2; } done; done)
USERLIBSRCS = $(shell test -z "$(ULIBPATHS)" || for d in $(ULIBPATHS); do for ss in $$d/*.cpp $$d/src/*.cpp; do test -r $$ss && echo $$ss; done; done)
INC_PATHS += $(USERLIBDIRS)
INC_PATHS += -I$(INODIR)/..
CPP_OBJECTS_CORE_EMU = $(CPP_SOURCES_CORE_EMU:.cpp=.cpp.o) $(USERLIBSRCS:.cpp=.cpp.o)
bin/fullcore.a: $(C_OBJECTS) $(CPP_OBJECTS_CORE_EMU)
$(VERBAR) ar -rcu $@ $^
$(VERBAR) ranlib -c $@
%: %.ino.cpp.o bin/fullcore.a
$(VERBLD) $(CXX) $(LDFLAGS) $< bin/fullcore.a $(LIBSSL) -o $@
@echo "----> $@ <----"
#################################################
# are we in primary make call ?
ifeq ($(INO),)
%: %.ino
@# recursive 'make' with paths
$(MAKE) -f $(MAKEFILE) MKFLAGS=-Wextra INODIR=$(dir $@) INO=$(notdir $@) $(BINDIR)/$(notdir $@)/$(notdir $@)
@# see below the new build rule with fixed output path outside from core location
#####################
# recursive call on ino target
else
$(BINDIR)/$(INO)/$(INO).ino.cpp:
@# arduino builder would come around here (.ino -> .ino.cpp)
@mkdir -p $(BINDIR)/$(INO); \
( \
echo "#include <$(INODIR)/$(INO).ino>"; \
for i in $(INODIR)/*.ino; do \
test "$$i" = $(INODIR)/$(INO).ino || echo "#include \"$$i\""; \
done; \
) > $(BINDIR)/$(INO)/$(INO).ino.cpp
endif # recursive
#####################
#################################################
.PHONY: list
list: # show core example list
@for dir in ../../libraries/*/examples/* \
../../libraries/*/examples/*/*; do \
test -d $$dir || continue; \
examplename=$${dir##*/}; \
test -f $${dir}/$${examplename}.ino || continue; \
echo $${dir}/$${examplename}; \
done | sort; \
#################################################
# help
.PHONY: help
help:
@cat README.txt
@echo ""
@echo "Make rules:"
@echo ""
@sed -rne 's,([^: \t]*):[^=#]*#[\t ]*(.*),\1 - \2,p' $(MAKEFILE)
@echo ""