1
0
mirror of https://github.com/esp8266/Arduino.git synced 2025-07-23 08:45:22 +03:00
Files
esp8266/tests/host/Makefile
Earle F. Philhower, III a389a995fb Add LittleFS as an optional filesystem, API compatible w/SPIFFS (but not on-flash-format compatible) (#5511)
* Add LittleFS as internal flash filesystem

Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem,
https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard
flash utilizing the exact same API as the existing SPIFFS filesystem.

LittleFS is built for low memory systems that are subject to random power
losses, is actively supported by the ARMmbed community, supports directories,
and seems to be much faster in the large-ish read-mostly applications I use.

LittleFS, however, has a larger minimum file allocation unit and does not do
static wear levelling.  This means that for systems that need many little
files (<4K), have small SPIFFS areas (64K), or which have a large static
set of files covering the majority of flash coupled with a frequently
updated set of other files, it may not perform as well.

Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch,
use LittleFS.open in place of SPIFFS.open to open files, and everything
else just works thanks to the magic of @igrr's File base class.

**LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS**
Since it is a completely different filesystem, you will need to reformat
your flash (and lose any data therein) to use it. Tools to build the
flash filesystem and upload are at
https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and
https://github.com/earlephilhower/mklittlefs/ .  The mklittlefs tool
is installed as part of the Arduino platform installation, automatically.

The included example shows a contrived read-mostly example and
demonstrates how the same calls work on either SPIFFS.* or LittleFS.*
Host tests are also included as part of CI.

Directories are fully supported in LittleFS. This means that LittleFS
will have a slight difference vs. SPIFFS when you use
LittleFS.openDir()/Dir.next().  On SPIFFS dir.next()
will return all filesystem entries, including ones in "subdirs"
(because in SPIFFS there are no subdirs and "/" is the same as any
other character in a filename).

On LittleFS, dir.next() will only return entries in the directory
specified, not subdirs.  So to list files in "/subdir/..." you need
to actually openDir("/subdir") and use Dir.next() to parse through
just those elements.  The returned filenames also only have the
filename returned, not full paths.  So on a FS with "/a/1", "/a/2"
when you do openDir("/a"); dir.next().getName(); you get "1" and "2"
and not "/a/1" and "/a/2" like in SPIFFS.  This is consistent with
POSIX ideas about reading directories and more natural for a FS.

Most code will not be affected by this, but if you depend on
openDir/Dir.next() you need to be aware of it.

Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile,
::openNextFile, and ::rewind methods added to Filesystem objects.
Documentation has been updated with this and other LittleFS information.

Subdirectories are made silently when they do not exist when you
try and create a file in a subdir.  They are silently removed when
the last file in them is deleted.  This is consistent with what
SPIFFS does but is obviously not normal POSIX behavior.  Since there
has never been a "FS.mkdir()" method this is the only way to be
compatible with legacy SPIFFS code.

SPIFFS code has been refactored to pull out common flash_hal_* ops
and placed in its own namespace, like LittleFS.

* Fix up merge blank line issue

* Merge in the FSConfig changs from SDFS PR

Enable setConfig for LittleFS as well plys merge the SPIFFS changes
done in the SDFS PR.

* Fix merge errors

* Update to use v2-alpha branch

The V2-alpha branch supports small file optimizations which can help
increase the utilization of flash when small files are prevalent.
It also adds support for metadata, which means we can start adding
things like file creation times, if desired (not yet).

* V2 of littlefs is now in upstream/master

* Update test to support non-creation-ordered files

In a directory, the order in which "readNextFile()" will return a name
is undefined.  SPIFFS may return it in order, but LittleFS does not as
of V2.  Update the test to look for files by name when doing
readNextFile() testing.

* Fix LittleFS.truncate implementation

* Fix SDFS tests

SDFS, SPIFFS, and LittleFS now all share the same common set of tests,
greatly increasing the SDFS test coverage.

* Update to point to mklittlefs v2

Upgrade mklittlefs to V2 format support

* Remove extra FS::write(const char *s) method

This was removed in #5861 and erroneously re-introduced here.

* Minimize spurious differences from master

* Dramatically reduce memory usage

Reduce the program and read chunk sizes which impacts performance
minimally but reduces per-file RAM usage of 16KB to <1KB.

* Add @d-a-v's host emulation for LittleFS

* Fix SW Serial library version

* Fix free space reporting

Thanks to @TD-er for discovering the issue

* Update littlefs to latest upstream

* Remove sdfat version included by accident

* Update SDFAT to include MOCK changes required

* Update to include SD.h test of file append
2019-05-25 09:53:24 +02:00

370 lines
9.6 KiB
Makefile

BINDIR := bin
LCOV_DIRECTORY := lcov
OUTPUT_BINARY := $(BINDIR)/host_tests
CORE_PATH := ../../cores/esp8266
LIBRARIES_PATH := ../../libraries
FORCE32 ?= 1
OPTZ ?= -Os
V ?= 0
MAKEFILE = $(word 1, $(MAKEFILE_LIST))
# I wasn't able to build with clang when -coverage flag is enabled, forcing GCC on OS X
ifeq ($(shell uname -s),Darwin)
CC ?= gcc
CXX ?= g++
endif
GCOV ?= gcov
VALGRIND ?= valgrind
LCOV ?= lcov
GENHTML ?= genhtml
ifeq ($(FORCE32),1)
SIZEOFLONG = $(shell echo 'int main(){return sizeof(long);}'|$(CXX) -m32 -x c++ - -o sizeoflong 2>/dev/null && ./sizeoflong; echo $$?; rm -f sizeoflong;)
ifneq ($(SIZEOFLONG),4)
$(warning Cannot compile in 32 bit mode, switching to native mode)
else
N32 = 32
M32 = -m32
endif
endif
ifeq ($(N32),32)
$(warning compiling in 32 bits mode)
else
$(warning compiling in native mode)
endif
ifeq ($(V), 0)
VERBC = @echo "C $@";
VERBCXX = @echo "C++ $@";
VERBLD = @echo "LD $@";
VERBAR = @echo "AR $@";
else
VERBC =
VERBCXX =
VERBLD =
VERBAR =
endif
$(shell mkdir -p $(BINDIR))
CORE_CPP_FILES := $(addprefix $(CORE_PATH)/,\
StreamString.cpp \
Stream.cpp \
WString.cpp \
Print.cpp \
FS.cpp \
spiffs_api.cpp \
MD5Builder.cpp \
../../libraries/LittleFS/src/LittleFS.cpp \
core_esp8266_noniso.cpp \
spiffs/spiffs_cache.cpp \
spiffs/spiffs_check.cpp \
spiffs/spiffs_gc.cpp \
spiffs/spiffs_hydrogen.cpp \
spiffs/spiffs_nucleus.cpp \
libb64/cencode.cpp \
libb64/cdecode.cpp \
Schedule.cpp \
HardwareSerial.cpp \
) \
$(addprefix $(LIBRARIES_PATH)/ESP8266SdFat/src/, \
FatLib/FatFile.cpp \
FatLib/FatFileLFN.cpp \
FatLib/FatFilePrint.cpp \
FatLib/FatFileSFN.cpp \
FatLib/FatVolume.cpp \
FatLib/FmtNumber.cpp \
FatLib/StdioStream.cpp \
) \
$(LIBRARIES_PATH)/SDFS/src/SDFS.cpp \
$(LIBRARIES_PATH)/SD/src/SD.cpp
CORE_C_FILES := $(addprefix $(CORE_PATH)/,\
../../libraries/LittleFS/src/lfs.c \
../../libraries/LittleFS/src/lfs_util.c \
)
MOCK_CPP_FILES_COMMON := $(addprefix common/,\
Arduino.cpp \
flash_hal_mock.cpp \
spiffs_mock.cpp \
littlefs_mock.cpp \
sdfs_mock.cpp \
WMath.cpp \
MockUART.cpp \
MockTools.cpp \
MocklwIP.cpp \
)
MOCK_CPP_FILES := $(MOCK_CPP_FILES_COMMON) $(addprefix common/,\
ArduinoCatch.cpp \
)
MOCK_CPP_FILES_EMU := $(MOCK_CPP_FILES_COMMON) $(addprefix common/,\
ArduinoMain.cpp \
ArduinoMainUdp.cpp \
ArduinoMainSpiffs.cpp \
ArduinoMainLittlefs.cpp \
user_interface.cpp \
)
MOCK_C_FILES := $(addprefix common/,\
md5.c \
noniso.c \
)
INC_PATHS += $(addprefix -I, \
. \
common \
$(CORE_PATH) \
)
INC_PATHS += $(addprefix -I,\
$(shell echo ../../libraries/*/src) \
$(shell echo ../../libraries/*) \
../../tools/sdk/include \
../../tools/sdk/lwip2/include \
)
TEST_CPP_FILES := \
fs/test_fs.cpp \
core/test_pgmspace.cpp \
core/test_md5builder.cpp \
core/test_string.cpp \
core/test_PolledTimeout.cpp \
core/test_Print.cpp
PREINCLUDES := \
-include common/mock.h \
-include common/c_types.h \
ifneq ($(D),)
OPTZ=-O0
DEBUG += -DDEBUG_ESP_PORT=Serial
DEBUG += -DDEBUG_ESP_SSL -DDEBUG_ESP_TLS_MEM -DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_HTTP_SERVER -DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_UPDATER -DDEBUG_ESP_OTA -DDEBUG_ESP_MDNS
endif
FLAGS += $(DEBUG) -Wall -coverage $(OPTZ) -fno-common -g $(M32)
FLAGS += -DHTTPCLIENT_1_1_COMPATIBLE=0
FLAGS += -DLWIP_IPV6=0
FLAGS += -DHOST_MOCK=1
FLAGS += -DNONOSDK221=1
FLAGS += $(MKFLAGS)
CXXFLAGS += -std=c++11 -fno-rtti $(FLAGS) -funsigned-char
CFLAGS += -std=c99 $(FLAGS) -funsigned-char
LDFLAGS += -coverage $(OPTZ) -g $(M32)
VALGRINDFLAGS += --leak-check=full --track-origins=yes --error-limit=no --show-leak-kinds=all --error-exitcode=999
CXXFLAGS += -Wno-error=format-security # cores/esp8266/Print.cpp:42:24: error: format not a string literal and no format arguments [-Werror=format-security] -- (os_printf_plus(not_the_best_way))
#CXXFLAGS += -Wno-format-security # cores/esp8266/Print.cpp:42:40: warning: format not a string literal and no format arguments [-Wformat-security] -- (os_printf_plus(not_the_best_way))
remduplicates = $(strip $(if $1,$(firstword $1) $(call remduplicates,$(filter-out $(firstword $1),$1))))
C_SOURCE_FILES = $(MOCK_C_FILES) $(CORE_C_FILES)
CPP_SOURCE_FILES = $(MOCK_CPP_FILES) $(CORE_CPP_FILES) $(TEST_CPP_FILES)
C_OBJECTS = $(C_SOURCE_FILES:.c=.c.o)
CPP_OBJECTS_CORE = $(MOCK_CPP_FILES:.cpp=.cpp.o) $(CORE_CPP_FILES:.cpp=.cpp.o)
CPP_OBJECTS_TESTS = $(TEST_CPP_FILES:.cpp=.cpp.o)
CPP_OBJECTS = $(CPP_OBJECTS_CORE) $(CPP_OBJECTS_TESTS)
OBJECTS = $(C_OBJECTS) $(CPP_OBJECTS)
COVERAGE_FILES = $(OBJECTS:.o=.gc*)
all: help
CI: # run CI
$(MAKE) -f $(MAKEFILE) MKFLAGS=-Werror FORCE32=0 OPTZ=-O0 doCI
doCI: build-info $(OUTPUT_BINARY) valgrind test gcov
test: $(OUTPUT_BINARY) # run host test for CI
$(OUTPUT_BINARY)
clean: clean-objects clean-coverage # clean everything
rm -rf $(BINDIR)
clean-objects:
rm -rf $(C_OBJECTS) $(CPP_OBJECTS_CORE) $(CPP_OBJECTS_CORE_EMU) $(CPP_OBJECTS_TESTS)
clean-coverage:
rm -rf $(COVERAGE_FILES) $(LCOV_DIRECTORY) *.gcov
gcov: test # run coverage for CI
find $(CORE_PATH) -name "*.gcno" -exec $(GCOV) -r -pb {} +
valgrind: $(OUTPUT_BINARY)
mkdir -p $(LCOV_DIRECTORY)
$(LCOV) --directory ../../cores/esp8266/ --zerocounters
$(VALGRIND) $(VALGRINDFLAGS) $(OUTPUT_BINARY)
$(LCOV) --directory $(CORE_PATH) --capture --output-file $(LCOV_DIRECTORY)/app.info
-$(GENHTML) $(LCOV_DIRECTORY)/app.info -o $(LCOV_DIRECTORY)
build-info: # show toolchain version
@echo "-------- build tools info --------"
@echo "CC: " $(CC)
$(CC) -v
@echo "CXX: " $(CXX)
$(CXX) -v
@echo "GCOV: " $(GCOV)
$(GCOV) -v
@echo "----------------------------------"
-include $(BINDIR)/.*.d
.SUFFIXES:
%.c.o: %.c
$(VERBC) $(CC) $(PREINCLUDES) $(CFLAGS) $(INC_PATHS) -MD -MF $(BINDIR)/.$(notdir $<).d -c -o $@ $<
.PRECIOUS: %.cpp.o
%.cpp.o: %.cpp
$(VERBCXX) $(CXX) $(PREINCLUDES) $(CXXFLAGS) $(INC_PATHS) -MD -MF $(BINDIR)/.$(notdir $<).d -c -o $@ $<
$(BINDIR)/core.a: $(C_OBJECTS) $(CPP_OBJECTS_CORE)
ar -rcu $@ $^
ranlib -c $@
$(OUTPUT_BINARY): $(CPP_OBJECTS_TESTS) $(BINDIR)/core.a
$(VERBLD) $(CXX) $(LDFLAGS) $^ -o $@
#################################################
# building ino sources
ARDUINO_LIBS := \
$(addprefix $(CORE_PATH)/,\
IPAddress.cpp \
Updater.cpp \
base64.cpp \
) \
$(addprefix ../../libraries/ESP8266WiFi/src/,\
ESP8266WiFi.cpp \
ESP8266WiFiAP.cpp \
ESP8266WiFiGeneric.cpp \
ESP8266WiFiMulti.cpp \
ESP8266WiFiSTA-WPS.cpp \
ESP8266WiFiSTA.cpp \
ESP8266WiFiScan.cpp \
WiFiClient.cpp \
WiFiUdp.cpp \
WiFiClientSecureBearSSL.cpp \
WiFiServerSecureBearSSL.cpp \
BearSSLHelpers.cpp \
CertStoreBearSSL.cpp \
)
OPT_ARDUINO_LIBS ?= $(addprefix ../../libraries/,\
$(addprefix ESP8266WebServer/src/,\
ESP8266WebServer.cpp \
Parsing.cpp \
detail/mimetable.cpp \
) \
$(addprefix ESP8266mDNS/src/,\
LEAmDNS.cpp \
LEAmDNS_Control.cpp \
LEAmDNS_Helpers.cpp \
LEAmDNS_Structs.cpp \
LEAmDNS_Transfer.cpp \
ESP8266mDNS.cpp \
) \
ArduinoOTA/ArduinoOTA.cpp \
DNSServer/src/DNSServer.cpp \
ESP8266AVRISP/src/ESP8266AVRISP.cpp \
ESP8266HTTPClient/src/ESP8266HTTPClient.cpp \
)
MOCK_ARDUINO_LIBS := $(addprefix common/,\
ClientContextSocket.cpp \
ClientContextTools.cpp \
MockWiFiServerSocket.cpp \
MockWiFiServer.cpp \
UdpContextSocket.cpp \
HostWiring.cpp \
MockEsp.cpp \
MockEEPROM.cpp \
MockSPI.cpp \
)
CPP_SOURCES_CORE_EMU = \
$(MOCK_CPP_FILES_EMU) \
$(CORE_CPP_FILES) \
$(MOCK_ARDUINO_LIBS) \
$(OPT_ARDUINO_LIBS) \
$(ARDUINO_LIBS) \
LIBSSLFILE = ../../tools/sdk/ssl/bearssl/build$(N32)/libbearssl.a
ifeq (,$(wildcard $(LIBSSLFILE)))
LIBSSL =
else
LIBSSL = $(LIBSSLFILE)
endif
ssl: # download source and build BearSSL
cd ../../tools/sdk/ssl && make native$(N32)
ULIBPATHS = $(shell echo $(ULIBDIRS) | sed 's,:, ,g')
USERLIBDIRS = $(shell test -z "$(ULIBPATHS)" || for d in $(ULIBPATHS); do for dd in $$d $$d/src; do test -d $$dd && { echo -I$$dd; echo "userlib: using directory '$$dd'" 1>&2; } done; done)
USERLIBSRCS = $(shell test -z "$(ULIBPATHS)" || for d in $(ULIBPATHS); do for ss in $$d/*.cpp $$d/src/*.cpp; do test -r $$ss && echo $$ss; done; done)
INC_PATHS += $(USERLIBDIRS)
INC_PATHS += -I$(INODIR)/..
CPP_OBJECTS_CORE_EMU = $(CPP_SOURCES_CORE_EMU:.cpp=.cpp.o) $(USERLIBSRCS:.cpp=.cpp.o)
bin/fullcore.a: $(C_OBJECTS) $(CPP_OBJECTS_CORE_EMU)
$(VERBAR) ar -rcu $@ $^
$(VERBAR) ranlib -c $@
%: %.ino.cpp.o bin/fullcore.a
$(VERBLD) $(CXX) $(LDFLAGS) $< bin/fullcore.a $(LIBSSL) -o $@
@echo "----> $@ <----"
#################################################
# are we in primary make call ?
ifeq ($(INO),)
%: %.ino
@# recursive 'make' with paths
$(MAKE) -f $(MAKEFILE) MKFLAGS=-Wextra INODIR=$(dir $@) INO=$(notdir $@) $(BINDIR)/$(notdir $@)/$(notdir $@)
@# see below the new build rule with fixed output path outside from core location
#####################
# recursive call on ino target
else
$(BINDIR)/$(INO)/$(INO).ino.cpp:
@# arduino builder would come around here (.ino -> .ino.cpp)
@mkdir -p $(BINDIR)/$(INO); \
( \
echo "#include <$(INODIR)/$(INO).ino>"; \
for i in $(INODIR)/*.ino; do \
test "$$i" = $(INODIR)/$(INO).ino || echo "#include \"$$i\""; \
done; \
) > $(BINDIR)/$(INO)/$(INO).ino.cpp
endif # recursive
#####################
#################################################
.PHONY: list
list: # show core example list
@for dir in ../../libraries/*/examples/* \
../../libraries/*/examples/*/*; do \
test -d $$dir || continue; \
examplename=$${dir##*/}; \
test -f $${dir}/$${examplename}.ino || continue; \
echo $${dir}/$${examplename}; \
done | sort; \
#################################################
# help
.PHONY: help
help:
@cat README.txt
@echo ""
@echo "Make rules:"
@echo ""
@sed -rne 's,([^: \t]*):[^=#]*#[\t ]*(.*),\1 - \2,p' $(MAKEFILE)
@echo ""