1
0
mirror of https://github.com/esp8266/Arduino.git synced 2025-04-22 21:23:07 +03:00
esp8266/doc/filesystem.rst
Earle F. Philhower, III b1da9eda46
SD Filesystem compatible with 8266 File, using latest SdFat (#5525)
* Add a FAT filesystem for SD cards to Arduino FS

Arduino forked a copy of SD lib several years ago, put their own wrapper
around it, and it's been languishing in our ESP8266 libraries ever since
as SD. It doesn't support long file names, has class names which
conflict with the ESP8266 internal names, and hasn't been updated in
ages.

The original author of the SD library has continued work in the
meantime, and produced a very feature rich implementation of SdFat. It
unfortunately also conflicts with the class names we use in ESP8266
Arduino and has a different API than the internal SPIFFS or proposed
LittleFS filesystem objects.

This PR puts a wrapper around the latest and greatest SdFat library,
by forking it and wrapping its classes in a private namespace "sdfat,"
and making as thin a wrapper as possible around it to conform to
the ESP8266 FS, File, and Dir classes.

This PR also removes the Arduino SD.h class library and rewrites it
using the new SDFS filesystem to make everything in the ESP8266
Arduino core compatible with each other.

By doing so it lets us use a single interface for anything needing a
file instead of multiple ones (see SDWebServer and how a different
object is needed vs. one serving from SPIFFS even though the logic is
all the same). Same for BearSSL's CertStores and probably a few others
I've missed, cleaning up our code base significantly.

Like LittleFS, silently create directories when a file is created with
a subdirectory specifier ("/path/to/file.txt") if they do not yet exist.

Adds a blacklist of sketches to skip in the CI process (because SdFat
has many examples which do not build properly on the ESP8266).

Now that LittleFS and SDFS have directory support, the FS needs to be
able to communicate whether a name is one or the other.  Add a simple
bool FS::isDirectory() and bool FS::isFile() method.  SPIFFS doesn't
have directories, so if it's valid it's a file and reported as such.

Add ::mkdir/::rmdir to the FS class to allow users to make and destroy
subdirectories.  SPIFFS directory operations will, of course, fail
and return false.

Emulate a 16MB SD card and allow test runner to exercise it by using
a custom SdFat HOST_MOCK-enabled object.

Throw out the original Arduino SD.h class and rewrite from scratch using
only the ESP8266 native SDFS calls.  This makes "SD" based applications
compatible with normal ESP8266 "File" and "FS" and "SPIFFS" operations.

The only major visible change for users is that long filenames now are
fully supported and work without any code changes.  If there are static
arrays of 11 bytes for old 8.3 names in code, they will need to be
adjusted.

While it is recommended to use the more powerful SDFS class to access SD
cards, this SD.h wrapper allows for use of existing Arduino libraries
which are built to only with with that SD class.

Additional helper functions added to ESP8266 native Filesystem:: classes
to help support this portability.

The rewrite is good enough to run the original SDWebServer and SD
example code without any changes.

* Add a FSConfig and SDFSConfig param to FS.begin()

Allows for configuration values to be passed into a filesystem via the
begin method.  By default, a FS will receive a nullptr and should so
whatever is appropriate.

The base FSConfig class has one parameter, _autoFormat, set by the
default constructor to true.

For SPIFFS, you can now disable auto formatting on mount failure by
passing in a FSConfig(false) object.

For SDFS a SDFSConfig parameter can be passed into config specifying the
chip select and SPI configuration.  If nothing is passed in, the begin
will fail since there are no safe default values here.

* Add FS::setConfig to set FS-specific options

Add a new call, FS::setConfig(const {SDFS,SPIFFS}Config *cfg), which
takes a FS-specific configuration object and copies any special settings
on a per-FS basis.  The call is only valid on unmounted filesystems, and
checks the type of object passed in matches the FS being configured.

Updates the docs and tests to utilize this new configuration method.

* Add ::truncate to File interface

Fixes #3846

* Use polledTimeout for formatting yields, cleanup

Use the new polledTimeout class to ensure a yield every 5ms while
formatting.

Add in default case handling and some debug messages when invalid inputs
specified.

* Make setConfig take const& ref, cleaner code

setConfig now can take a parameter defined directly in the call by using
a const &ref to it, leading to one less line of code to write and
cleaner reading of the code.

Also clean up SDFS implementation pointer definition.
2019-03-06 02:14:44 +00:00

13 KiB

Filesystem

Flash layout

Even though file system is stored on the same flash chip as the program, programming new sketch will not modify file system contents. This allows to use file system to store sketch data, configuration files, or content for Web server.

The following diagram illustrates flash layout used in Arduino environment:

|--------------|-------|---------------|--|--|--|--|--|
^              ^       ^               ^     ^
Sketch    OTA update   File system   EEPROM  WiFi config (SDK)

File system size depends on the flash chip size. Depending on the board which is selected in IDE, you have the following options for flash size:

Board Flash chip size, bytes File system size, bytes
Generic module 512k 64k, 128k
Generic module 1M 64k, 128k, 256k, 512k
Generic module 2M 1M
Generic module 4M 1M, 2M, 3M
Adafruit HUZZAH 4M 1M, 2M, 3M
ESPresso Lite 1.0 4M 1M, 2M, 3M
ESPresso Lite 2.0 4M 1M, 2M, 3M
NodeMCU 0.9 4M 1M, 2M, 3M
NodeMCU 1.0 4M 1M, 2M, 3M
Olimex MOD-WIFI-ESP8266(-DEV) 2M 1M
SparkFun Thing 512k 64k
SweetPea ESP-210 4M 1M, 2M, 3M
WeMos D1 R1, R2 & mini 4M 1M, 2M, 3M
ESPDuino 4M 1M, 2M, 3M
WiFiduino 4M 1M, 2M, 3M

Note: to use any of file system functions in the sketch, add the following include to the sketch:

#include "FS.h"

File system limitations

The filesystem implementation for ESP8266 had to accomodate the constraints of the chip, among which its limited RAM. SPIFFS was selected because it is designed for small systems, but that comes at the cost of some simplifications and limitations.

First, behind the scenes, SPIFFS does not support directories, it just stores a "flat" list of files. But contrary to traditional filesystems, the slash character '/' is allowed in filenames, so the functions that deal with directory listing (e.g. openDir("/website")) basically just filter the filenames and keep the ones that start with the requested prefix (/website/). Practically speaking, that makes little difference though.

Second, there is a limit of 32 chars in total for filenames. One '\0' char is reserved for C string termination, so that leaves us with 31 usable characters.

Combined, that means it is advised to keep filenames short and not use deeply nested directories, as the full path of each file (including directories, '/' characters, base name, dot and extension) has to be 31 chars at a maximum. For example, the filename /website/images/bird_thumbnail.jpg is 34 chars and will cause some problems if used, for example in exists() or in case another file starts with the same first 31 characters.

Warning: That limit is easily reached and if ignored, problems might go unnoticed because no error message will appear at compilation nor runtime.

For more details on the internals of SPIFFS implementation, see the SPIFFS readme file.

Uploading files to file system

ESP8266FS is a tool which integrates into the Arduino IDE. It adds a menu item to Tools menu for uploading the contents of sketch data directory into ESP8266 flash file system.

  • Download the tool: https://github.com/esp8266/arduino-esp8266fs-plugin/releases/download/0.3.0/ESP8266FS-0.3.0.zip.
  • In your Arduino sketchbook directory, create tools directory if it doesn't exist yet
  • Unpack the tool into tools directory (the path will look like <home_dir>/Arduino/tools/ESP8266FS/tool/esp8266fs.jar)
  • Restart Arduino IDE
  • Open a sketch (or create a new one and save it)
  • Go to sketch directory (choose Sketch > Show Sketch Folder)
  • Create a directory named data and any files you want in the file system there
  • Make sure you have selected a board, port, and closed Serial Monitor
  • Select Tools > ESP8266 Sketch Data Upload. This should start uploading the files into ESP8266 flash file system. When done, IDE status bar will display SPIFFS Image Uploaded message.

File system object (SPIFFS)

setConfig

SPIFFSConfig cfg;
cfg.setAutoFormat(false);
SPIFFS.setConfig(cfg);

This method allows you to configure the parameters of a filesystem before mounting. All filesystems have their own *Config (i.e. SDFSConfig or SPIFFSConfig with their custom set of options. All filesystems allow explicitly enabling/disabling formatting when mounts fail. If you do not call this setConfig method before perforing begin(), you will get the filesystem's default behavior and configuration. By default, SPIFFS will autoformat the filesystem if it cannot mount it, while SDFS will not.

begin

SPIFFS.begin()

This method mounts SPIFFS file system. It must be called before any other FS APIs are used. Returns true if file system was mounted successfully, false otherwise. With no options it will format SPIFFS if it is unable to mount it on the first try.

end

SPIFFS.end()

This method unmounts SPIFFS file system. Use this method before updating SPIFFS using OTA.

format

SPIFFS.format()

Formats the file system. May be called either before or after calling begin. Returns true if formatting was successful.

open

SPIFFS.open(path, mode)

Opens a file. path should be an absolute path starting with a slash (e.g. /dir/filename.txt). mode is a string specifying access mode. It can be one of "r", "w", "a", "r+", "w+", "a+". Meaning of these modes is the same as for fopen C function.

r      Open text file for reading.  The stream is positioned at the
       beginning of the file.

r+     Open for reading and writing.  The stream is positioned at the
       beginning of the file.

w      Truncate file to zero length or create text file for writing.
       The stream is positioned at the beginning of the file.

w+     Open for reading and writing.  The file is created if it does
       not exist, otherwise it is truncated.  The stream is
       positioned at the beginning of the file.

a      Open for appending (writing at end of file).  The file is
       created if it does not exist.  The stream is positioned at the
       end of the file.

a+     Open for reading and appending (writing at end of file).  The
       file is created if it does not exist.  The initial file
       position for reading is at the beginning of the file, but
       output is always appended to the end of the file.

Returns File object. To check whether the file was opened successfully, use the boolean operator.

File f = SPIFFS.open("/f.txt", "w");
if (!f) {
    Serial.println("file open failed");
}

exists

SPIFFS.exists(path)

Returns true if a file with given path exists, false otherwise.

openDir

SPIFFS.openDir(path)

Opens a directory given its absolute path. Returns a Dir object.

remove

SPIFFS.remove(path)

Deletes the file given its absolute path. Returns true if file was deleted successfully.

rename

SPIFFS.rename(pathFrom, pathTo)

Renames file from pathFrom to pathTo. Paths must be absolute. Returns true if file was renamed successfully.

info

FSInfo fs_info;
SPIFFS.info(fs_info);

Fills FSInfo structure with information about the file system. Returns true is successful, false otherwise.

Filesystem information structure

struct FSInfo {
    size_t totalBytes;
    size_t usedBytes;
    size_t blockSize;
    size_t pageSize;
    size_t maxOpenFiles;
    size_t maxPathLength;
};

This is the structure which may be filled using FS::info method. -totalBytes — total size of useful data on the file system -usedBytes — number of bytes used by files - blockSize — SPIFFS block size - pageSize — SPIFFS logical page size - maxOpenFiles — max number of files which may be open simultaneously -maxPathLength — max file name length (including one byte for zero termination)

Directory object (Dir)

The purpose of Dir object is to iterate over files inside a directory. It provides the methods: next(), fileName(), fileSize() , and openFile(mode).

The following example shows how it should be used:

Dir dir = SPIFFS.openDir("/data");
while (dir.next()) {
    Serial.print(dir.fileName());
    if(dir.fileSize()) {
        File f = dir.openFile("r");
        Serial.println(f.size());
    }
}

next

Returns true while there are files in the directory to iterate over. It must be called before calling fileName(), fileSize(), and openFile() functions.

fileName

Returns the name of the current file pointed to by the internal iterator.

fileSize

Returns the size of the current file pointed to by the internal iterator.

openFile

This method takes mode argument which has the same meaning as for SPIFFS.open() function.

File object

SPIFFS.open() and dir.openFile() functions return a File object. This object supports all the functions of Stream, so you can use readBytes, findUntil, parseInt, println, and all other Stream methods.

There are also some functions which are specific to File object.

seek

file.seek(offset, mode)

This function behaves like fseek C function. Depending on the value of mode, it moves current position in a file as follows:

  • if mode is SeekSet, position is set to offset bytes from the beginning.
  • if mode is SeekCur, current position is moved by offset bytes.
  • if mode is SeekEnd, position is set to offset bytes from the end of the file.

Returns true if position was set successfully.

position

file.position()

Returns the current position inside the file, in bytes.

size

file.size()

Returns file size, in bytes.

name

String name = file.name();

Returns file name, as const char*. Convert it to String for storage.

close

file.close()

Close the file. No other operations should be performed on File object after close function was called.