2 Commits

Author	SHA1	Message	Date
Earle F. Philhower, III	a389a995fb	Add LittleFS as an optional filesystem, API compatible w/SPIFFS (but not on-flash-format compatible) (#5511) ... * Add LittleFS as internal flash filesystem Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem, https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard flash utilizing the exact same API as the existing SPIFFS filesystem. LittleFS is built for low memory systems that are subject to random power losses, is actively supported by the ARMmbed community, supports directories, and seems to be much faster in the large-ish read-mostly applications I use. LittleFS, however, has a larger minimum file allocation unit and does not do static wear levelling. This means that for systems that need many little files (<4K), have small SPIFFS areas (64K), or which have a large static set of files covering the majority of flash coupled with a frequently updated set of other files, it may not perform as well. Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch, use LittleFS.open in place of SPIFFS.open to open files, and everything else just works thanks to the magic of @igrr's File base class. **LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS** Since it is a completely different filesystem, you will need to reformat your flash (and lose any data therein) to use it. Tools to build the flash filesystem and upload are at https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and https://github.com/earlephilhower/mklittlefs/ . The mklittlefs tool is installed as part of the Arduino platform installation, automatically. The included example shows a contrived read-mostly example and demonstrates how the same calls work on either SPIFFS.* or LittleFS.* Host tests are also included as part of CI. Directories are fully supported in LittleFS. This means that LittleFS will have a slight difference vs. SPIFFS when you use LittleFS.openDir()/Dir.next(). On SPIFFS dir.next() will return all filesystem entries, including ones in "subdirs" (because in SPIFFS there are no subdirs and "/" is the same as any other character in a filename). On LittleFS, dir.next() will only return entries in the directory specified, not subdirs. So to list files in "/subdir/..." you need to actually openDir("/subdir") and use Dir.next() to parse through just those elements. The returned filenames also only have the filename returned, not full paths. So on a FS with "/a/1", "/a/2" when you do openDir("/a"); dir.next().getName(); you get "1" and "2" and not "/a/1" and "/a/2" like in SPIFFS. This is consistent with POSIX ideas about reading directories and more natural for a FS. Most code will not be affected by this, but if you depend on openDir/Dir.next() you need to be aware of it. Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile, ::openNextFile, and ::rewind methods added to Filesystem objects. Documentation has been updated with this and other LittleFS information. Subdirectories are made silently when they do not exist when you try and create a file in a subdir. They are silently removed when the last file in them is deleted. This is consistent with what SPIFFS does but is obviously not normal POSIX behavior. Since there has never been a "FS.mkdir()" method this is the only way to be compatible with legacy SPIFFS code. SPIFFS code has been refactored to pull out common flash_hal_* ops and placed in its own namespace, like LittleFS. * Fix up merge blank line issue * Merge in the FSConfig changs from SDFS PR Enable setConfig for LittleFS as well plys merge the SPIFFS changes done in the SDFS PR. * Fix merge errors * Update to use v2-alpha branch The V2-alpha branch supports small file optimizations which can help increase the utilization of flash when small files are prevalent. It also adds support for metadata, which means we can start adding things like file creation times, if desired (not yet). * V2 of littlefs is now in upstream/master * Update test to support non-creation-ordered files In a directory, the order in which "readNextFile()" will return a name is undefined. SPIFFS may return it in order, but LittleFS does not as of V2. Update the test to look for files by name when doing readNextFile() testing. * Fix LittleFS.truncate implementation * Fix SDFS tests SDFS, SPIFFS, and LittleFS now all share the same common set of tests, greatly increasing the SDFS test coverage. * Update to point to mklittlefs v2 Upgrade mklittlefs to V2 format support * Remove extra FS::write(const char *s) method This was removed in #5861 and erroneously re-introduced here. * Minimize spurious differences from master * Dramatically reduce memory usage Reduce the program and read chunk sizes which impacts performance minimally but reduces per-file RAM usage of 16KB to <1KB. * Add @d-a-v's host emulation for LittleFS * Fix SW Serial library version * Fix free space reporting Thanks to @TD-er for discovering the issue * Update littlefs to latest upstream * Remove sdfat version included by accident * Update SDFAT to include MOCK changes required * Update to include SD.h test of file append	2019-05-25 09:53:24 +02:00
Earle F. Philhower, III	b1da9eda46	SD Filesystem compatible with 8266 File, using latest SdFat (#5525) ... * Add a FAT filesystem for SD cards to Arduino FS Arduino forked a copy of SD lib several years ago, put their own wrapper around it, and it's been languishing in our ESP8266 libraries ever since as SD. It doesn't support long file names, has class names which conflict with the ESP8266 internal names, and hasn't been updated in ages. The original author of the SD library has continued work in the meantime, and produced a very feature rich implementation of SdFat. It unfortunately also conflicts with the class names we use in ESP8266 Arduino and has a different API than the internal SPIFFS or proposed LittleFS filesystem objects. This PR puts a wrapper around the latest and greatest SdFat library, by forking it and wrapping its classes in a private namespace "sdfat," and making as thin a wrapper as possible around it to conform to the ESP8266 FS, File, and Dir classes. This PR also removes the Arduino SD.h class library and rewrites it using the new SDFS filesystem to make everything in the ESP8266 Arduino core compatible with each other. By doing so it lets us use a single interface for anything needing a file instead of multiple ones (see SDWebServer and how a different object is needed vs. one serving from SPIFFS even though the logic is all the same). Same for BearSSL's CertStores and probably a few others I've missed, cleaning up our code base significantly. Like LittleFS, silently create directories when a file is created with a subdirectory specifier ("/path/to/file.txt") if they do not yet exist. Adds a blacklist of sketches to skip in the CI process (because SdFat has many examples which do not build properly on the ESP8266). Now that LittleFS and SDFS have directory support, the FS needs to be able to communicate whether a name is one or the other. Add a simple bool FS::isDirectory() and bool FS::isFile() method. SPIFFS doesn't have directories, so if it's valid it's a file and reported as such. Add ::mkdir/::rmdir to the FS class to allow users to make and destroy subdirectories. SPIFFS directory operations will, of course, fail and return false. Emulate a 16MB SD card and allow test runner to exercise it by using a custom SdFat HOST_MOCK-enabled object. Throw out the original Arduino SD.h class and rewrite from scratch using only the ESP8266 native SDFS calls. This makes "SD" based applications compatible with normal ESP8266 "File" and "FS" and "SPIFFS" operations. The only major visible change for users is that long filenames now are fully supported and work without any code changes. If there are static arrays of 11 bytes for old 8.3 names in code, they will need to be adjusted. While it is recommended to use the more powerful SDFS class to access SD cards, this SD.h wrapper allows for use of existing Arduino libraries which are built to only with with that SD class. Additional helper functions added to ESP8266 native Filesystem:: classes to help support this portability. The rewrite is good enough to run the original SDWebServer and SD example code without any changes. * Add a FSConfig and SDFSConfig param to FS.begin() Allows for configuration values to be passed into a filesystem via the begin method. By default, a FS will receive a nullptr and should so whatever is appropriate. The base FSConfig class has one parameter, _autoFormat, set by the default constructor to true. For SPIFFS, you can now disable auto formatting on mount failure by passing in a FSConfig(false) object. For SDFS a SDFSConfig parameter can be passed into config specifying the chip select and SPI configuration. If nothing is passed in, the begin will fail since there are no safe default values here. * Add FS::setConfig to set FS-specific options Add a new call, FS::setConfig(const {SDFS,SPIFFS}Config *cfg), which takes a FS-specific configuration object and copies any special settings on a per-FS basis. The call is only valid on unmounted filesystems, and checks the type of object passed in matches the FS being configured. Updates the docs and tests to utilize this new configuration method. * Add ::truncate to File interface Fixes #3846 * Use polledTimeout for formatting yields, cleanup Use the new polledTimeout class to ensure a yield every 5ms while formatting. Add in default case handling and some debug messages when invalid inputs specified. * Make setConfig take const& ref, cleaner code setConfig now can take a parameter defined directly in the call by using a const &ref to it, leading to one less line of code to write and cleaner reading of the code. Also clean up SDFS implementation pointer definition.	2019-03-06 02:14:44 +00:00