1
0
mirror of https://github.com/esp8266/Arduino.git synced 2025-07-27 18:02:17 +03:00

Add LittleFS as an optional filesystem, API compatible w/SPIFFS (but not on-flash-format compatible) (#5511)

* Add LittleFS as internal flash filesystem

Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem,
https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard
flash utilizing the exact same API as the existing SPIFFS filesystem.

LittleFS is built for low memory systems that are subject to random power
losses, is actively supported by the ARMmbed community, supports directories,
and seems to be much faster in the large-ish read-mostly applications I use.

LittleFS, however, has a larger minimum file allocation unit and does not do
static wear levelling.  This means that for systems that need many little
files (<4K), have small SPIFFS areas (64K), or which have a large static
set of files covering the majority of flash coupled with a frequently
updated set of other files, it may not perform as well.

Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch,
use LittleFS.open in place of SPIFFS.open to open files, and everything
else just works thanks to the magic of @igrr's File base class.

**LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS**
Since it is a completely different filesystem, you will need to reformat
your flash (and lose any data therein) to use it. Tools to build the
flash filesystem and upload are at
https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and
https://github.com/earlephilhower/mklittlefs/ .  The mklittlefs tool
is installed as part of the Arduino platform installation, automatically.

The included example shows a contrived read-mostly example and
demonstrates how the same calls work on either SPIFFS.* or LittleFS.*
Host tests are also included as part of CI.

Directories are fully supported in LittleFS. This means that LittleFS
will have a slight difference vs. SPIFFS when you use
LittleFS.openDir()/Dir.next().  On SPIFFS dir.next()
will return all filesystem entries, including ones in "subdirs"
(because in SPIFFS there are no subdirs and "/" is the same as any
other character in a filename).

On LittleFS, dir.next() will only return entries in the directory
specified, not subdirs.  So to list files in "/subdir/..." you need
to actually openDir("/subdir") and use Dir.next() to parse through
just those elements.  The returned filenames also only have the
filename returned, not full paths.  So on a FS with "/a/1", "/a/2"
when you do openDir("/a"); dir.next().getName(); you get "1" and "2"
and not "/a/1" and "/a/2" like in SPIFFS.  This is consistent with
POSIX ideas about reading directories and more natural for a FS.

Most code will not be affected by this, but if you depend on
openDir/Dir.next() you need to be aware of it.

Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile,
::openNextFile, and ::rewind methods added to Filesystem objects.
Documentation has been updated with this and other LittleFS information.

Subdirectories are made silently when they do not exist when you
try and create a file in a subdir.  They are silently removed when
the last file in them is deleted.  This is consistent with what
SPIFFS does but is obviously not normal POSIX behavior.  Since there
has never been a "FS.mkdir()" method this is the only way to be
compatible with legacy SPIFFS code.

SPIFFS code has been refactored to pull out common flash_hal_* ops
and placed in its own namespace, like LittleFS.

* Fix up merge blank line issue

* Merge in the FSConfig changs from SDFS PR

Enable setConfig for LittleFS as well plys merge the SPIFFS changes
done in the SDFS PR.

* Fix merge errors

* Update to use v2-alpha branch

The V2-alpha branch supports small file optimizations which can help
increase the utilization of flash when small files are prevalent.
It also adds support for metadata, which means we can start adding
things like file creation times, if desired (not yet).

* V2 of littlefs is now in upstream/master

* Update test to support non-creation-ordered files

In a directory, the order in which "readNextFile()" will return a name
is undefined.  SPIFFS may return it in order, but LittleFS does not as
of V2.  Update the test to look for files by name when doing
readNextFile() testing.

* Fix LittleFS.truncate implementation

* Fix SDFS tests

SDFS, SPIFFS, and LittleFS now all share the same common set of tests,
greatly increasing the SDFS test coverage.

* Update to point to mklittlefs v2

Upgrade mklittlefs to V2 format support

* Remove extra FS::write(const char *s) method

This was removed in #5861 and erroneously re-introduced here.

* Minimize spurious differences from master

* Dramatically reduce memory usage

Reduce the program and read chunk sizes which impacts performance
minimally but reduces per-file RAM usage of 16KB to <1KB.

* Add @d-a-v's host emulation for LittleFS

* Fix SW Serial library version

* Fix free space reporting

Thanks to @TD-er for discovering the issue

* Update littlefs to latest upstream

* Remove sdfat version included by accident

* Update SDFAT to include MOCK changes required

* Update to include SD.h test of file append
This commit is contained in:
Earle F. Philhower, III
2019-05-25 00:53:24 -07:00
committed by david gauchard
parent b55199227b
commit a389a995fb
67 changed files with 2115 additions and 513 deletions

View File

@ -57,6 +57,7 @@ CORE_CPP_FILES := $(addprefix $(CORE_PATH)/,\
FS.cpp \
spiffs_api.cpp \
MD5Builder.cpp \
../../libraries/LittleFS/src/LittleFS.cpp \
core_esp8266_noniso.cpp \
spiffs/spiffs_cache.cpp \
spiffs/spiffs_check.cpp \
@ -81,11 +82,15 @@ CORE_CPP_FILES := $(addprefix $(CORE_PATH)/,\
$(LIBRARIES_PATH)/SD/src/SD.cpp
CORE_C_FILES := $(addprefix $(CORE_PATH)/,\
../../libraries/LittleFS/src/lfs.c \
../../libraries/LittleFS/src/lfs_util.c \
)
MOCK_CPP_FILES_COMMON := $(addprefix common/,\
Arduino.cpp \
flash_hal_mock.cpp \
spiffs_mock.cpp \
littlefs_mock.cpp \
sdfs_mock.cpp \
WMath.cpp \
MockUART.cpp \
@ -101,6 +106,7 @@ MOCK_CPP_FILES_EMU := $(MOCK_CPP_FILES_COMMON) $(addprefix common/,\
ArduinoMain.cpp \
ArduinoMainUdp.cpp \
ArduinoMainSpiffs.cpp \
ArduinoMainLittlefs.cpp \
user_interface.cpp \
)

View File

@ -44,6 +44,7 @@
bool user_exit = false;
const char* host_interface = nullptr;
size_t spiffs_kb = 1024;
size_t littlefs_kb = 1024;
bool ignore_sigint = false;
bool restore_tty = false;
bool mockdebug = false;
@ -127,9 +128,10 @@ void help (const char* argv0, int exitcode)
" -f - no throttle (possibly 100%%CPU)\n"
" -b - blocking tty/mocked-uart (default: not blocking tty)\n"
" -S - spiffs size in KBytes (default: %zd)\n"
" -L - littlefs size in KBytes (default: %zd)\n"
" -v - mock verbose\n"
" (negative value will force mismatched size)\n"
, argv0, MOCK_PORT_SHIFTER, spiffs_kb);
, argv0, MOCK_PORT_SHIFTER, spiffs_kb, littlefs_kb);
exit(exitcode);
}
@ -143,12 +145,14 @@ static struct option options[] =
{ "verbose", no_argument, NULL, 'v' },
{ "interface", required_argument, NULL, 'i' },
{ "spiffskb", required_argument, NULL, 'S' },
{ "littlefskb", required_argument, NULL, 'L' },
{ "portshifter", required_argument, NULL, 's' },
};
void cleanup ()
{
mock_stop_spiffs();
mock_stop_littlefs();
mock_stop_uart();
}
@ -204,6 +208,9 @@ int main (int argc, char* const argv [])
case 'S':
spiffs_kb = atoi(optarg);
break;
case 'L':
littlefs_kb = atoi(optarg);
break;
case 'b':
blocking_uart = true;
break;
@ -226,6 +233,15 @@ int main (int argc, char* const argv [])
mock_start_spiffs(name, spiffs_kb);
}
if (littlefs_kb)
{
String name = argv[0];
name += "-littlefs";
name += String(littlefs_kb > 0? littlefs_kb: -littlefs_kb, DEC);
name += "KB";
mock_start_littlefs(name, littlefs_kb);
}
// setup global global_ipv4_netfmt
wifi_get_ip_info(0, nullptr);

View File

@ -0,0 +1,36 @@
/* Emulate the flash read/write HAL */
#include <stdint.h>
#include <string.h>
extern "C"
{
uint32_t s_phys_addr = 0;
uint32_t s_phys_size = 0;
uint32_t s_phys_page = 0;
uint32_t s_phys_block = 0;
uint8_t* s_phys_data = nullptr;
}
int32_t flash_hal_read(uint32_t addr, uint32_t size, uint8_t *dst) {
memcpy(dst, s_phys_data + addr, size);
return 0;
}
int32_t flash_hal_write(uint32_t addr, uint32_t size, const uint8_t *src) {
memcpy(s_phys_data + addr, src, size);
return 0;
}
int32_t flash_hal_erase(uint32_t addr, uint32_t size) {
if ((size & (FLASH_SECTOR_SIZE - 1)) != 0 ||
(addr & (FLASH_SECTOR_SIZE - 1)) != 0) {
abort();
}
const uint32_t sector = addr / FLASH_SECTOR_SIZE;
const uint32_t sectorCount = size / FLASH_SECTOR_SIZE;
for (uint32_t i = 0; i < sectorCount; ++i) {
memset(s_phys_data + (sector + i) * FLASH_SECTOR_SIZE, 0xff, FLASH_SECTOR_SIZE);
}
return 0;
}

View File

@ -0,0 +1,19 @@
#ifndef flash_hal_mock_h
#define flash_hal_mock_h
#include <stdint.h>
extern "C"
{
extern uint32_t s_phys_addr;
extern uint32_t s_phys_size;
extern uint32_t s_phys_page;
extern uint32_t s_phys_block;
extern uint8_t* s_phys_data;
}
extern int32_t flash_hal_read(uint32_t addr, uint32_t size, uint8_t *dst);
extern int32_t flash_hal_write(uint32_t addr, uint32_t size, const uint8_t *src);
extern int32_t flash_hal_erase(uint32_t addr, uint32_t size);
#endif

View File

@ -0,0 +1,132 @@
/*
littlefs_mock.cpp - LittleFS mock for host side testing
Copyright © 2019 Earle F. Philhower, III
Based off spiffs_mock.cpp:
Copyright © 2016 Ivan Grokhotkov
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
*/
#include "littlefs_mock.h"
#include "spiffs_mock.h"
#include "spiffs/spiffs.h"
#include "debug.h"
#include <flash_utils.h>
#include <stdlib.h>
#include <LittleFS.h>
#include <spiffs_api.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include "flash_hal_mock.h"
#define LITTLEFS_FILE_NAME "littlefs.bin"
FS LittleFS(nullptr);
LittleFSMock::LittleFSMock(ssize_t fs_size, size_t fs_block, size_t fs_page, const String& storage)
{
m_storage = storage;
if ((m_overwrite = (fs_size < 0)))
fs_size = -fs_size;
fprintf(stderr, "LittleFS: %zd bytes\n", fs_size);
m_fs.resize(fs_size, 0xff);
s_phys_addr = 0;
s_phys_size = static_cast<uint32_t>(fs_size);
s_phys_page = static_cast<uint32_t>(fs_page);
s_phys_block = static_cast<uint32_t>(fs_block);
s_phys_data = m_fs.data();
reset();
}
void LittleFSMock::reset()
{
LittleFS = FS(FSImplPtr(new littlefs_impl::LittleFSImpl(0, s_phys_size, s_phys_page, s_phys_block, 5)));
load();
}
LittleFSMock::~LittleFSMock()
{
save();
s_phys_addr = 0;
s_phys_size = 0;
s_phys_page = 0;
s_phys_block = 0;
s_phys_data = nullptr;
m_fs.resize(0);
LittleFS = FS(FSImplPtr(nullptr));
}
void LittleFSMock::load ()
{
if (!m_fs.size() || !m_storage.length())
return;
int fs = ::open(m_storage.c_str(), O_RDONLY);
if (fs == -1)
{
fprintf(stderr, "LittleFS: loading '%s': %s\n", m_storage.c_str(), strerror(errno));
return;
}
off_t flen = lseek(fs, 0, SEEK_END);
if (flen == (off_t)-1)
{
fprintf(stderr, "LittleFS: checking size of '%s': %s\n", m_storage.c_str(), strerror(errno));
return;
}
lseek(fs, 0, SEEK_SET);
if (flen != (off_t)m_fs.size())
{
fprintf(stderr, "LittleFS: size of '%s': %d does not match requested size %zd\n", m_storage.c_str(), (int)flen, m_fs.size());
if (!m_overwrite)
{
fprintf(stderr, "LittleFS: aborting at user request\n");
exit(1);
}
fprintf(stderr, "LittleFS: continuing without loading at user request, '%s' will be overwritten\n", m_storage.c_str());
}
else
{
fprintf(stderr, "LittleFS: loading %zi bytes from '%s'\n", m_fs.size(), m_storage.c_str());
ssize_t r = ::read(fs, m_fs.data(), m_fs.size());
if (r != (ssize_t)m_fs.size())
fprintf(stderr, "LittleFS: reading %zi bytes: returned %zd: %s\n", m_fs.size(), r, strerror(errno));
}
::close(fs);
}
void LittleFSMock::save ()
{
if (!m_fs.size() || !m_storage.length())
return;
int fs = ::open(m_storage.c_str(), O_CREAT | O_TRUNC | O_WRONLY, 0644);
if (fs == -1)
{
fprintf(stderr, "LittleFS: saving '%s': %s\n", m_storage.c_str(), strerror(errno));
return;
}
fprintf(stderr, "LittleFS: saving %zi bytes to '%s'\n", m_fs.size(), m_storage.c_str());
if (::write(fs, m_fs.data(), m_fs.size()) != (ssize_t)m_fs.size())
fprintf(stderr, "LittleFS: writing %zi bytes: %s\n", m_fs.size(), strerror(errno));
if (::close(fs) == -1)
fprintf(stderr, "LittleFS: closing %s: %s\n", m_storage.c_str(), strerror(errno));
}

View File

@ -0,0 +1,48 @@
/*
littlefs_mock.h - LittleFS HAL mock for host side testing
Copyright © 2019 Earle F. Philhower, III
Based on spiffs_mock:
Copyright © 2016 Ivan Grokhotkov
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
*/
#ifndef littlefs_mock_hpp
#define littlefs_mock_hpp
#include <stdint.h>
#include <stddef.h>
#include <vector>
#include <FS.h>
#include "flash_hal_mock.h"
#define DEFAULT_LITTLEFS_FILE_NAME "littlefs.bin"
class LittleFSMock {
public:
LittleFSMock(ssize_t fs_size, size_t fs_block, size_t fs_page, const String& storage = emptyString);
void reset();
~LittleFSMock();
protected:
void load ();
void save ();
std::vector<uint8_t> m_fs;
String m_storage;
bool m_overwrite;
};
#define LITTLEFS_MOCK_DECLARE(size_kb, block_kb, page_b, storage) LittleFSMock littlefs_mock(size_kb * 1024, block_kb * 1024, page_b, storage)
#define LITTLEFS_MOCK_RESET() littlefs_mock.reset()
#endif /* littlefs_mock_hpp */

View File

@ -135,6 +135,8 @@ void register_udp (int sock, UdpContext* udp = nullptr);
void mock_start_spiffs (const String& fname, size_t size_kb, size_t block_kb = 8, size_t page_b = 512);
void mock_stop_spiffs ();
void mock_start_littlefs (const String& fname, size_t size_kb, size_t block_kb = 8, size_t page_b = 512);
void mock_stop_littlefs ();
//

View File

@ -17,5 +17,5 @@
#include "../../../libraries/SDFS/src/SDFS.h"
#define SDSIZE 16LL
uint64_t _sdCardSizeB = SDSIZE * 1024LL * 1024LL;
uint8_t _sdCard[SDSIZE * 1024LL * 1024LL];
uint64_t _sdCardSizeB = 0;
uint8_t *_sdCard = nullptr;

View File

@ -23,11 +23,21 @@
class SDFSMock {
public:
SDFSMock() { }
SDFSMock(ssize_t fs_size, size_t fs_block, size_t fs_page, const String& storage = emptyString) { }
void reset() { }
~SDFSMock() { }
};
#define SDFS_MOCK_DECLARE() SDFSMock sdfs_mock();
extern uint64_t _sdCardSizeB;
extern uint8_t *_sdCard;
#define SDFS_MOCK_DECLARE(size_kb, block_kb, page_b, storage) \
SDFS.end(); \
SDFSMock sdfs_mock(size_kb * 1024, block_kb * 1024, page_b, storage); free(_sdCard); \
_sdCardSizeB = size_kb ? 16 * 1024 * 1024 : 0; \
if (_sdCardSizeB) _sdCard = (uint8_t*)calloc(_sdCardSizeB, 1); \
else _sdCard = nullptr; \
SDFS.setConfig(SDFSConfig().setAutoFormat(true));
#define SDFS_MOCK_RESET() sdfs_mock.reset()
#endif /* spiffs_mock_hpp */

View File

@ -27,14 +27,9 @@
#include <fcntl.h>
#include <unistd.h>
extern "C"
{
static uint32_t s_phys_addr = 0;
uint32_t s_phys_size = 0;
uint32_t s_phys_page = 0;
uint32_t s_phys_block = 0;
uint8_t* s_phys_data = nullptr;
}
#include "flash_hal_mock.h"
#define SPIFFS_FILE_NAME "spiffs.bin"
FS SPIFFS(nullptr);
@ -57,7 +52,7 @@ SpiffsMock::SpiffsMock(ssize_t fs_size, size_t fs_block, size_t fs_page, const S
void SpiffsMock::reset()
{
SPIFFS = FS(FSImplPtr(new SPIFFSImpl(0, s_phys_size, s_phys_page, s_phys_block, 5)));
SPIFFS = FS(FSImplPtr(new spiffs_impl::SPIFFSImpl(0, s_phys_size, s_phys_page, s_phys_block, 5)));
load();
}
@ -131,26 +126,3 @@ void SpiffsMock::save ()
if (::close(fs) == -1)
fprintf(stderr, "SPIFFS: closing %s: %s\n", m_storage.c_str(), strerror(errno));
}
int32_t spiffs_hal_read(uint32_t addr, uint32_t size, uint8_t *dst) {
memcpy(dst, s_phys_data + addr, size);
return SPIFFS_OK;
}
int32_t spiffs_hal_write(uint32_t addr, uint32_t size, uint8_t *src) {
memcpy(s_phys_data + addr, src, size);
return SPIFFS_OK;
}
int32_t spiffs_hal_erase(uint32_t addr, uint32_t size) {
if ((size & (FLASH_SECTOR_SIZE - 1)) != 0 ||
(addr & (FLASH_SECTOR_SIZE - 1)) != 0) {
abort();
}
const uint32_t sector = addr / FLASH_SECTOR_SIZE;
const uint32_t sectorCount = size / FLASH_SECTOR_SIZE;
for (uint32_t i = 0; i < sectorCount; ++i) {
memset(s_phys_data + (sector + i) * FLASH_SECTOR_SIZE, 0xff, FLASH_SECTOR_SIZE);
}
return SPIFFS_OK;
}

View File

@ -20,6 +20,7 @@
#include <stddef.h>
#include <vector>
#include <FS.h>
#include "flash_hal_mock.h"
#define DEFAULT_SPIFFS_FILE_NAME "spiffs.bin"

View File

@ -17,39 +17,29 @@
#include <map>
#include <FS.h>
#include "../common/spiffs_mock.h"
#include "../common/littlefs_mock.h"
#include "../common/sdfs_mock.h"
#include <spiffs/spiffs.h>
#include <LittleFS.h>
#include "../../../libraries/SDFS/src/SDFS.h"
#include "../../../libraries/SD/src/SD.h"
static void createFile (const char* name, const char* content)
{
auto f = SPIFFS.open(name, "w");
REQUIRE(f);
if (content) {
f.print(content);
}
}
static String readFile (const char* name)
{
auto f = SPIFFS.open(name, "r");
if (f) {
return f.readString();
}
return String();
}
static std::set<String> listDir (const char* path)
{
std::set<String> result;
Dir dir = SPIFFS.openDir(path);
while (dir.next()) {
REQUIRE(result.find(dir.fileName()) == std::end(result));
result.insert(dir.fileName());
}
return result;
}
namespace spiffs_test {
#define FSTYPE SPIFFS
#define TESTPRE "SPIFFS - "
#define TESTPAT "[fs]"
#define TOOLONGFILENAME "/2345678901234567890123456789012"
#define FS_MOCK_DECLARE SPIFFS_MOCK_DECLARE
#define FS_MOCK_RESET SPIFFS_MOCK_RESET
#undef FS_HAS_DIRS
#include "test_fs.inc"
#undef FSTYPE
#undef TESTPRE
#undef TESTPAT
#undef TOOLONGFILENAME
#undef FS_MOCK_DECLARE
#undef FS_MOCK_RESET
TEST_CASE("SPIFFS checks the config object passed in", "[fs]")
{
@ -57,286 +47,90 @@ TEST_CASE("SPIFFS checks the config object passed in", "[fs]")
FSConfig f;
SPIFFSConfig s;
SDFSConfig d;
LittleFSConfig l;
REQUIRE_FALSE(SPIFFS.setConfig(f));
REQUIRE(SPIFFS.setConfig(s));
REQUIRE_FALSE(SPIFFS.setConfig(d));
REQUIRE_FALSE(LittleFS.setConfig(l));
}
TEST_CASE("SDFS checks the config object passed in", "[fs]")
};
namespace littlefs_test {
#define FSTYPE LittleFS
#define TESTPRE "LittleFS - "
#define TESTPAT "[lfs]"
// LittleFS routines strip leading slashes before doing anything, so up to 31 char names are allowable
#define TOOLONGFILENAME "/12345678901234567890123456789012"
#define FS_MOCK_DECLARE LITTLEFS_MOCK_DECLARE
#define FS_MOCK_RESET LITTLEFS_MOCK_RESET
#define FS_HAS_DIRS
#include "test_fs.inc"
#undef FSTYPE
#undef TESTPRE
#undef TESTPAT
#undef TOOLONGFILENAME
#undef FS_MOCK_DECLARE
#undef FS_MOCK_RESET
TEST_CASE("LittleFS checks the config object passed in", "[fs]")
{
SDFS_MOCK_DECLARE();
LITTLEFS_MOCK_DECLARE(64, 8, 512, "");
FSConfig f;
SPIFFSConfig s;
SDFSConfig d;
LittleFSConfig l;
REQUIRE_FALSE(LittleFS.setConfig(f));
REQUIRE_FALSE(LittleFS.setConfig(s));
REQUIRE_FALSE(LittleFS.setConfig(d));
REQUIRE(LittleFS.setConfig(l));
}
};
namespace sdfs_test {
#define FSTYPE SDFS
#define TESTPRE "SDFS - "
#define TESTPAT "[sdfs]"
// SDFS supports long paths (MAXPATH)
#define TOOLONGFILENAME "/" \
"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890" \
"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890" \
"12345678901234567890123456789012345678901234567890123456"
#define FS_MOCK_DECLARE SDFS_MOCK_DECLARE
#define FS_MOCK_RESET SDFS_MOCK_RESET
#define FS_HAS_DIRS
#include "test_fs.inc"
#undef FSTYPE
#undef TESTPRE
#undef TESTPAT
#undef TOOLONGFILENAME
#undef FS_MOCK_DECLARE
#undef FS_MOCK_RESET
TEST_CASE("SDFS checks the config object passed in", "[fs]")
{
SDFS_MOCK_DECLARE(64, 8, 512, "");
FSConfig f;
SPIFFSConfig s;
SDFSConfig d;
LittleFSConfig l;
REQUIRE_FALSE(SDFS.setConfig(f));
REQUIRE_FALSE(SDFS.setConfig(s));
REQUIRE(SDFS.setConfig(d));
REQUIRE_FALSE(SDFS.setConfig(l));
}
TEST_CASE("FS can begin","[fs]")
// Also a SD specific test to check that FILE_OPEN is really an append operation:
TEST_CASE("SD.h FILE_WRITE macro is append", "[fs]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
SPIFFSConfig cfg;
cfg.setAutoFormat(false);
SPIFFS.setConfig(cfg);
REQUIRE_FALSE(SPIFFS.begin());
cfg.setAutoFormat(true);
SPIFFS.setConfig(cfg);
REQUIRE(SPIFFS.begin());
REQUIRE_FALSE(SPIFFS.setConfig(cfg)); // Can't change config of mounted FS
}
TEST_CASE("FS can't begin with zero size","[fs]")
{
SPIFFS_MOCK_DECLARE(0, 8, 512, "");
REQUIRE_FALSE(SPIFFS.begin());
}
TEST_CASE("Before begin is called, open will fail","[fs]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE_FALSE(SPIFFS.open("/foo", "w"));
}
TEST_CASE("FS can create file","[fs]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(SPIFFS.begin());
createFile("/test", "");
REQUIRE(SPIFFS.exists("/test"));
}
TEST_CASE("Files can be written and appended to","[fs]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(SPIFFS.begin());
{
File f = SPIFFS.open("config1.txt", "w");
REQUIRE(f);
f.println("file 1");
}
{
File f = SPIFFS.open("config1.txt", "a");
REQUIRE(f);
f.println("file 1 again");
}
{
File f = SPIFFS.open("config1.txt", "r");
REQUIRE(f);
char buf[128];
size_t len = f.read((uint8_t*)buf, sizeof(buf));
buf[len] = 0;
REQUIRE(strcmp(buf, "file 1\r\nfile 1 again\r\n") == 0);
}
}
TEST_CASE("Files persist after reset", "[fs]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(SPIFFS.begin());
createFile("config1.txt", "file 1");
SPIFFS_MOCK_RESET();
REQUIRE(SPIFFS.begin());
REQUIRE(readFile("config1.txt") == "file 1");
}
TEST_CASE("Filesystem is empty after format", "[fs]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(SPIFFS.format());
REQUIRE(SPIFFS.begin());
createFile("/1", "first");
createFile("/2", "second");
REQUIRE(SPIFFS.format());
Dir root = SPIFFS.openDir("/");
size_t count = 0;
while (root.next()) {
++count;
}
REQUIRE(count == 0);
}
TEST_CASE("Dir lists all files", "[fs]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(SPIFFS.begin());
createFile("/empty", "");
createFile("/not_empty", "some text");
createFile("/another", "more text");
createFile("/subdir/empty", "");
createFile("/subdir/not_empty", "text again");
auto files = listDir("/");
REQUIRE(files.size() == 5);
REQUIRE(files.find("/empty") != std::end(files));
REQUIRE(files.find("/not_empty") != std::end(files));
REQUIRE(files.find("/another") != std::end(files));
REQUIRE(files.find("/subdir/empty") != std::end(files));
REQUIRE(files.find("/subdir/not_empty") != std::end(files));
}
TEST_CASE("File names which are too long are rejected", "[fs]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(SPIFFS.begin());
const char* emptyName = "";
const char* longName_31 = "/234567890123456789012345678901";
const char* longName_32 = "/2345678901234567890123456789012";
REQUIRE_FALSE(SPIFFS.open(emptyName, "w"));
REQUIRE_FALSE(SPIFFS.open(emptyName, "r"));
REQUIRE_FALSE(SPIFFS.exists(emptyName));
REQUIRE_FALSE(SPIFFS.open(longName_32, "w"));
REQUIRE_FALSE(SPIFFS.open(longName_32, "r"));
REQUIRE_FALSE(SPIFFS.exists(longName_32));
REQUIRE(SPIFFS.open(longName_31, "w"));
REQUIRE(SPIFFS.open(longName_31, "r"));
REQUIRE(SPIFFS.exists(longName_31));
}
TEST_CASE("#1685 Duplicate files", "[fs][bugreport]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(SPIFFS.begin());
createFile("/config", "some text");
createFile("/data", "");
readFile("/config");
createFile("/data", "more text");
listDir("/");
}
TEST_CASE("#1819 Can list all files with openDir(\"\")", "[fs][bugreport]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(SPIFFS.begin());
createFile("/file1", "some text");
createFile("/file2", "other text");
createFile("file3", "more text");
createFile("sorta-dir/file4", "\n");
auto files = listDir("");
REQUIRE(files.size() == 4);
}
TEST_CASE("truncate", "[fs][spiffs]")
{
SPIFFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(SPIFFS.begin());
createFile("/file1", "some text");
auto f = SPIFFS.open("/file1", "r");
f.truncate(4);
f.close();
String s = readFile("/file1");
REQUIRE( s == "some" );
}
TEST_CASE("SDFS", "[sdfs]")
{
SDFS_MOCK_DECLARE();
auto cfg = SDFSConfig(0, SD_SCK_MHZ(1));
SDFS.setConfig(cfg);
REQUIRE(SDFS.format());
SDFS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(SDFS.begin());
REQUIRE_FALSE(SDFS.setConfig(cfg)); // Can't change config of mounted fs
REQUIRE(SDFS.mkdir("/happy/face"));
REQUIRE(SDFS.mkdir("/happy/nose"));
REQUIRE(SDFS.rmdir("/happy/face"));
auto f = SDFS.open("/this/is/a/long/name.txt", "w");
f.printf("Hello\n");
f.close();
SDFS.end();
}
TEST_CASE("Files.ino example", "[sd]")
{
SDFS_MOCK_DECLARE();
SDFS.end();
SDFS.setConfig(SDFSConfig(0, SD_SCK_MHZ(1)));
REQUIRE(SDFS.format());
REQUIRE(SD.begin(4));
REQUIRE_FALSE(SD.exists("example.txt"));
File myFile = SD.open("example.txt", FILE_WRITE);
REQUIRE(myFile);
myFile.close();
REQUIRE(SD.exists("example.txt"));
REQUIRE(SD.remove("example.txt"));
REQUIRE_FALSE(SD.exists("example.txt"));
SDFS.end();
}
static void createFileSD(const char* name, const char* content)
{
auto f = SD.open(name, FILE_WRITE);
REQUIRE(f);
if (content) {
f.print(content);
}
}
static String readFileSD(const char* name)
{
auto f = SD.open(name, FILE_READ);
if (f) {
return f.readString();
}
return String();
}
TEST_CASE("Listfiles.ino example", "[sd]")
{
SDFS_MOCK_DECLARE();
SDFS.setConfig(SDFSConfig(0, SD_SCK_MHZ(1)));
REQUIRE(SDFS.format());
REQUIRE(SD.begin(4));
createFileSD("file1", "hello");
createFileSD("file2", "hola");
createFileSD("dir1/file3", "nihao");
createFileSD("dir2/dir3/file4", "bonjour");
File root = SD.open("/");
File file1 = root.openNextFile();
File file2 = root.openNextFile();
File dir1 = root.openNextFile();
File dir1_file3 = dir1.openNextFile();
File dir2 = root.openNextFile();
File dir2_dir3 = dir2.openNextFile();
File dir2_dir3_file4 = dir2_dir3.openNextFile();
bool ok;
ok = root.isDirectory() && !root.isFile() && !strcmp(root.name(), "/");
REQUIRE(ok);
ok = !file1.isDirectory() && file1.isFile() && !strcmp(file1.name(), "file1");
REQUIRE(ok);
ok = !file2.isDirectory() && file2.isFile() && !strcmp(file2.name(), "file2");
REQUIRE(ok);
ok = dir1.isDirectory() && !dir1.isFile() && !strcmp(dir1.name(), "dir1");
REQUIRE(ok);
ok = !dir1_file3.isDirectory() && dir1_file3.isFile() && !strcmp(dir1_file3.name(), "file3") &&
!strcmp(dir1_file3.fullName(), "dir1/file3");
REQUIRE(ok);
ok = dir2.isDirectory() && !dir2.isFile() && !strcmp(dir2.name(), "dir2");
REQUIRE(ok);
ok = dir2_dir3.isDirectory() && !dir2_dir3.isFile() && !strcmp(dir2_dir3.name(), "dir3");
REQUIRE(ok);
ok = !dir2_dir3_file4.isDirectory() && dir2_dir3_file4.isFile() && !strcmp(dir2_dir3_file4.name(), "file4") &&
!strcmp(dir2_dir3_file4.fullName(), "dir2/dir3/file4");
REQUIRE(ok);
REQUIRE(readFileSD("/file1") == "hello");
REQUIRE(readFileSD("file2") == "hola");
REQUIRE(readFileSD("dir1/file3") == "nihao");
REQUIRE(readFileSD("/dir2/dir3/file4") == "bonjour");
}
TEST_CASE("Multisplendored File::writes", "[fs]")
{
SDFS_MOCK_DECLARE();
SDFS.end();
SDFS.setConfig(SDFSConfig(0, SD_SCK_MHZ(1)));
REQUIRE(SDFS.format());
REQUIRE(SD.begin(4));
File f = SD.open("/file.txt", FILE_WRITE);
@ -349,7 +143,11 @@ TEST_CASE("Multisplendored File::writes", "[fs]")
uint32_t bigone = 0x40404040;
f.write((const uint8_t*)&bigone, 4);
f.close();
REQUIRE(readFileSD("/file.txt") == "aAbbcctheendxyz@@@@");
REQUIRE(readFile("/file.txt") == "aAbbcctheendxyz@@@@");
f = SD.open("/file.txt", FILE_WRITE);
f.write("append", 6);
f.close();
REQUIRE(readFile("/file.txt") == "aAbbcctheendxyz@@@@append");
File g = SD.open("/file2.txt", FILE_WRITE);
g.write(0);
g.close();
@ -359,3 +157,5 @@ TEST_CASE("Multisplendored File::writes", "[fs]")
g.close();
REQUIRE(u == 0);
}
};

346
tests/host/fs/test_fs.inc Normal file
View File

@ -0,0 +1,346 @@
static void createFile (const char* name, const char* content)
{
auto f = FSTYPE.open(name, "w");
REQUIRE(f);
if (content) {
f.print(content);
}
}
static String readFile (const char* name)
{
auto f = FSTYPE.open(name, "r");
if (f) {
return f.readString();
}
return String();
}
static std::set<String> listDir (const char* path)
{
std::set<String> result;
Dir dir = FSTYPE.openDir(path);
while (dir.next()) {
REQUIRE(result.find(dir.fileName()) == std::end(result));
result.insert(dir.fileName());
}
return result;
}
TEST_CASE(TESTPRE "FS can begin",TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.begin());
}
TEST_CASE(TESTPRE "FS can't begin with zero size",TESTPAT)
{
FS_MOCK_DECLARE(0, 8, 512, "");
REQUIRE_FALSE(FSTYPE.begin());
}
TEST_CASE(TESTPRE "Before begin is called, open will fail",TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE_FALSE(FSTYPE.open("/foo", "w"));
}
TEST_CASE(TESTPRE "FS can create file",TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.begin());
createFile("/test", "");
REQUIRE(FSTYPE.exists("/test"));
}
TEST_CASE(TESTPRE "Files can be written and appended to",TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.begin());
{
File f = FSTYPE.open("config1.txt", "w");
REQUIRE(f);
f.println("file 1");
}
{
File f = FSTYPE.open("config1.txt", "a");
REQUIRE(f);
f.println("file 1 again");
}
{
File f = FSTYPE.open("config1.txt", "r");
REQUIRE(f);
char buf[128];
size_t len = f.read((uint8_t*)buf, sizeof(buf));
buf[len] = 0;
REQUIRE(strcmp(buf, "file 1\r\nfile 1 again\r\n") == 0);
}
}
TEST_CASE(TESTPRE "Files persist after reset", TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.begin());
createFile("config1.txt", "file 1");
FS_MOCK_RESET();
REQUIRE(FSTYPE.begin());
REQUIRE(readFile("config1.txt") == "file 1");
}
TEST_CASE(TESTPRE "Filesystem is empty after format", TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.format());
REQUIRE(FSTYPE.begin());
createFile("/1", "first");
createFile("/2", "second");
FSTYPE.end();
REQUIRE(FSTYPE.format());
REQUIRE(FSTYPE.begin());
Dir root = FSTYPE.openDir("/");
size_t count = 0;
while (root.next()) {
++count;
}
REQUIRE(count == 0);
}
TEST_CASE(TESTPRE "File names which are too long are rejected", TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.begin());
const char* emptyName = "";
const char* longName_31 = "/234567890123456789012345678901";
const char* longName_32 = TOOLONGFILENAME;
REQUIRE_FALSE(FSTYPE.open(emptyName, "w"));
REQUIRE_FALSE(FSTYPE.open(emptyName, "r"));
REQUIRE_FALSE(FSTYPE.exists(emptyName));
REQUIRE_FALSE(FSTYPE.open(longName_32, "w"));
REQUIRE_FALSE(FSTYPE.open(longName_32, "r"));
REQUIRE_FALSE(FSTYPE.exists(longName_32));
REQUIRE(FSTYPE.open(longName_31, "w"));
REQUIRE(FSTYPE.open(longName_31, "r"));
REQUIRE(FSTYPE.exists(longName_31));
}
TEST_CASE(TESTPRE "#1685 Duplicate files", "[fs][bugreport]")
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.begin());
createFile("/config", "some text");
createFile("/data", "");
readFile("/config");
createFile("/data", "more text");
auto files = listDir("/");
REQUIRE(files.size() == 2);
}
TEST_CASE(TESTPRE "#1819 Can list all files with openDir(\"\")", "[fs][bugreport]")
{
FS_MOCK_DECLARE(96, 8, 512, "");
REQUIRE(FSTYPE.begin());
createFile("/file1", "some text");
createFile("/file2", "other text");
createFile("file3", "more text");
createFile("sorta-dir/file4", "\n");
auto files = listDir("");
REQUIRE(files.size() == 4);
}
TEST_CASE(TESTPRE "truncate", TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.begin());
createFile("/file1", "some text");
auto f = FSTYPE.open("/file1", "r+");
REQUIRE(f.truncate(4));
f.close();
String s = readFile("/file1");
REQUIRE( s == "some" );
}
#ifdef FS_HAS_DIRS
#if FSTYPE != SDFS
// We silently make subdirectories if they do not exist and silently remove
// them when they're no longer needed, so make sure we can clean up after
// ourselves. At some point we may drop this and go to normal POSIX mkdir
// behavior and expose the FS::mkdir() method, but for now this works OK.
TEST_CASE(TESTPRE "Removing all files in a subdir removes that subdir", TESTPAT)
{
FS_MOCK_DECLARE(128, 8, 512, "");
REQUIRE(FSTYPE.begin());
createFile("/empty", "");
createFile("/not_empty", "some text");
createFile("/another", "more text");
createFile("/subdir/empty", "");
createFile("/subdir/not_empty", "text again");
auto files = listDir("/");
REQUIRE(files.size() == 4);
files = listDir("/subdir");
REQUIRE(files.size() == 2);
// Delete one of subdir, should still exist afterwards
FSTYPE.remove("subdir/empty");
files = listDir("/subdir");
REQUIRE(files.size() == 1);
FSTYPE.remove("subdir/not_empty");
files = listDir("/subdir");
REQUIRE(files.size() == 0);
files = listDir("/");
REQUIRE(files.size() == 3);
REQUIRE(files.find("subdir") == std::end(files));
}
#endif
// LittleFS openDir is slightly different than SPIFFS. In SPIFFS there
// are no directories and "/" is just another character, so "/a/b/c" is a
// file in the root dir whose name is "/a/b/c". In LittleFS we have full
// directory support, so "/a/b/c" is a file "c" in the "/a/b" dir.
// This means that if you iterate over dirOpen("/") on SPIFFS you get
// a list of every file, including "subdirs". On LittleFS, you need to
// explicitly open the subdir to see its files. This behavior is the
// same as POSIX readdir(), and helps isolate subdirs from each other.
// Also note that the returned filenames in the "dir.next()" operator
// will be in that subdir (i.e. if you opendir("/a/b"); f=dir.next();"
// f.name == "c" and not "/a/b/c" as you would see in SPIFFS.
TEST_CASE(TESTPRE "Dir lists all files", TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.begin());
createFile("/empty", "");
createFile("/not_empty", "some text");
createFile("/another", "more text");
createFile("/subdir/empty", "");
createFile("/subdir/not_empty", "text again");
auto files = listDir("/");
REQUIRE(files.size() == 4);
bool empty = (files.find("/empty") != std::end(files)) || (files.find("empty") != std::end(files));
REQUIRE(empty);
bool not_empty = (files.find("/not_empty") != std::end(files)) || (files.find("not_empty") != std::end(files));
REQUIRE(not_empty);
bool another = (files.find("/another") != std::end(files)) || (files.find("another") != std::end(files));
REQUIRE(another);
files = listDir("/subdir");
REQUIRE(files.size() == 2);
bool sub_empty = (files.find("/empty") != std::end(files)) || (files.find("empty") != std::end(files));
REQUIRE(sub_empty);
bool sub_not_empty = (files.find("/not_empty") != std::end(files)) || (files.find("not_empty") != std::end(files));
REQUIRE(sub_not_empty);
}
File FindFileByName(const File f[], int count, const char *name)
{
for (int i=0; i<count; i++) {
if (!strcmp(name, f[i].name())) return f[i];
}
return f[0];
}
TEST_CASE(TESTPRE "Listfiles.ino example", TESTPAT)
{
FS_MOCK_DECLARE(128, 8, 512, "");
REQUIRE(FSTYPE.format());
REQUIRE(FSTYPE.begin());
createFile("file1", "hello");
createFile("file2", "hola");
createFile("dir1/file3", "nihao");
createFile("dir2/dir3/file4", "bonjour");
File root = FSTYPE.open("/", "r");
// LittleFS and SDFS are not guaranteed to put the names in order of creation, so
// manually look for them...
File f[4];
f[0] = root.openNextFile();
f[1] = root.openNextFile();
f[2] = root.openNextFile();
f[3] = root.openNextFile();
File file1 = FindFileByName(f, 4, "file1");
File file2 = FindFileByName(f, 4, "file2");
File dir1 = FindFileByName(f, 4, "dir1");
File dir1_file3 = dir1.openNextFile();
File dir2 = FindFileByName(f, 4, "dir2");
File dir2_dir3 = dir2.openNextFile();
File dir2_dir3_file4 = dir2_dir3.openNextFile();
bool ok;
ok = root.isDirectory() && !root.isFile() && !strcmp(root.name(), "/");
REQUIRE(ok);
ok = !file1.isDirectory() && file1.isFile() && !strcmp(file1.name(), "file1");
REQUIRE(ok);
ok = !file2.isDirectory() && file2.isFile() && !strcmp(file2.name(), "file2");
REQUIRE(ok);
ok = dir1.isDirectory() && !dir1.isFile() && !strcmp(dir1.name(), "dir1");
REQUIRE(ok);
ok = !dir1_file3.isDirectory() && dir1_file3.isFile() && !strcmp(dir1_file3.name(), "file3") &&
!strcmp(dir1_file3.fullName(), "dir1/file3");
REQUIRE(ok);
ok = dir2.isDirectory() && !dir2.isFile() && !strcmp(dir2.name(), "dir2");
REQUIRE(ok);
ok = dir2_dir3.isDirectory() && !dir2_dir3.isFile() && !strcmp(dir2_dir3.name(), "dir3");
REQUIRE(ok);
ok = !dir2_dir3_file4.isDirectory() && dir2_dir3_file4.isFile() && !strcmp(dir2_dir3_file4.name(), "file4") &&
!strcmp(dir2_dir3_file4.fullName(), "dir2/dir3/file4");
REQUIRE(ok);
REQUIRE(readFile("/file1") == "hello");
REQUIRE(readFile("file2") == "hola");
REQUIRE(readFile("dir1/file3") == "nihao");
REQUIRE(readFile("/dir2/dir3/file4") == "bonjour");
}
#else // !FS_HAS_DIRS
TEST_CASE(TESTPRE "Dir lists all files", TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.begin());
createFile("/empty", "");
createFile("/not_empty", "some text");
createFile("/another", "more text");
createFile("/subdir/empty", "");
createFile("/subdir/not_empty", "text again");
auto files = listDir("/");
REQUIRE(files.size() == 5);
bool empty = (files.find("/empty") != std::end(files)) || (files.find("empty") != std::end(files));
REQUIRE(empty);
bool not_empty = (files.find("/not_empty") != std::end(files)) || (files.find("not_empty") != std::end(files));
REQUIRE(not_empty);
bool another = (files.find("/another") != std::end(files)) || (files.find("another") != std::end(files));
REQUIRE(another);
bool sub_empty = (files.find("/subdir/empty") != std::end(files)) || (files.find("subdir/empty") != std::end(files));
REQUIRE(sub_empty);
bool sub_not_empty = (files.find("/subdir/not_empty") != std::end(files)) || (files.find("subdir/not_empty") != std::end(files));
REQUIRE(sub_not_empty);
}
TEST_CASE(TESTPRE "Multisplendored File::writes", TESTPAT)
{
FS_MOCK_DECLARE(64, 8, 512, "");
REQUIRE(FSTYPE.begin());
File f = FSTYPE.open("/file.txt", "w");
f.write('a');
f.write(65);
f.write("bbcc");
f.write("theend", 6);
char block[3]={'x','y','z'};
f.write(block, 3);
uint32_t bigone = 0x40404040;
f.write((const uint8_t*)&bigone, 4);
f.close();
REQUIRE(readFile("/file.txt") == "aAbbcctheendxyz@@@@");
File g = FSTYPE.open("/file.txt", "w");
g.write(0);
g.close();
g = FSTYPE.open("/file.txt", "r");
uint8_t u = 0x66;
g.read(&u, 1);
g.close();
REQUIRE(u == 0);
}
#endif