arduino/esp8266
1
0
Fork 0
mirror of https://github.com/esp8266/Arduino.git synced 2025-08-15 19:22:45 +03:00
Code Activity

SD Filesystem compatible with 8266 File, using latest SdFat (#5525)

Browse Source 
* Add a FAT filesystem for SD cards to Arduino FS

Arduino forked a copy of SD lib several years ago, put their own wrapper
around it, and it's been languishing in our ESP8266 libraries ever since
as SD. It doesn't support long file names, has class names which
conflict with the ESP8266 internal names, and hasn't been updated in
ages.

The original author of the SD library has continued work in the
meantime, and produced a very feature rich implementation of SdFat. It
unfortunately also conflicts with the class names we use in ESP8266
Arduino and has a different API than the internal SPIFFS or proposed
LittleFS filesystem objects.

This PR puts a wrapper around the latest and greatest SdFat library,
by forking it and wrapping its classes in a private namespace "sdfat,"
and making as thin a wrapper as possible around it to conform to
the ESP8266 FS, File, and Dir classes.

This PR also removes the Arduino SD.h class library and rewrites it
using the new SDFS filesystem to make everything in the ESP8266
Arduino core compatible with each other.

By doing so it lets us use a single interface for anything needing a
file instead of multiple ones (see SDWebServer and how a different
object is needed vs. one serving from SPIFFS even though the logic is
all the same). Same for BearSSL's CertStores and probably a few others
I've missed, cleaning up our code base significantly.

Like LittleFS, silently create directories when a file is created with
a subdirectory specifier ("/path/to/file.txt") if they do not yet exist.

Adds a blacklist of sketches to skip in the CI process (because SdFat
has many examples which do not build properly on the ESP8266).

Now that LittleFS and SDFS have directory support, the FS needs to be
able to communicate whether a name is one or the other.  Add a simple
bool FS::isDirectory() and bool FS::isFile() method.  SPIFFS doesn't
have directories, so if it's valid it's a file and reported as such.

Add ::mkdir/::rmdir to the FS class to allow users to make and destroy
subdirectories.  SPIFFS directory operations will, of course, fail
and return false.

Emulate a 16MB SD card and allow test runner to exercise it by using
a custom SdFat HOST_MOCK-enabled object.

Throw out the original Arduino SD.h class and rewrite from scratch using
only the ESP8266 native SDFS calls.  This makes "SD" based applications
compatible with normal ESP8266 "File" and "FS" and "SPIFFS" operations.

The only major visible change for users is that long filenames now are
fully supported and work without any code changes.  If there are static
arrays of 11 bytes for old 8.3 names in code, they will need to be
adjusted.

While it is recommended to use the more powerful SDFS class to access SD
cards, this SD.h wrapper allows for use of existing Arduino libraries
which are built to only with with that SD class.

Additional helper functions added to ESP8266 native Filesystem:: classes
to help support this portability.

The rewrite is good enough to run the original SDWebServer and SD
example code without any changes.

* Add a FSConfig and SDFSConfig param to FS.begin()

Allows for configuration values to be passed into a filesystem via the
begin method.  By default, a FS will receive a nullptr and should so
whatever is appropriate.

The base FSConfig class has one parameter, _autoFormat, set by the
default constructor to true.

For SPIFFS, you can now disable auto formatting on mount failure by
passing in a FSConfig(false) object.

For SDFS a SDFSConfig parameter can be passed into config specifying the
chip select and SPI configuration.  If nothing is passed in, the begin
will fail since there are no safe default values here.

* Add FS::setConfig to set FS-specific options

Add a new call, FS::setConfig(const {SDFS,SPIFFS}Config *cfg), which
takes a FS-specific configuration object and copies any special settings
on a per-FS basis.  The call is only valid on unmounted filesystems, and
checks the type of object passed in matches the FS being configured.

Updates the docs and tests to utilize this new configuration method.

* Add ::truncate to File interface

Fixes #3846

* Use polledTimeout for formatting yields, cleanup

Use the new polledTimeout class to ensure a yield every 5ms while
formatting.

Add in default case handling and some debug messages when invalid inputs
specified.

* Make setConfig take const& ref, cleaner code

setConfig now can take a parameter defined directly in the call by using
a const &ref to it, leading to one less line of code to write and
cleaner reading of the code.

Also clean up SDFS implementation pointer definition.
This commit is contained in:
Earle F. Philhower, III
2019-03-06 02:14:44 +00:00
committed by GitHub
parent 61a8a6b14e
commit b1da9eda46
36 changed files with 1659 additions and 5537 deletions
Download Patch File Download Diff File Expand all files Collapse all files

627
libraries/SD/src/SD.cpp
View File

@@ -1,632 +1,5 @@
/*
SD - a slightly more friendly wrapper for sdfatlib
This library aims to expose a subset of SD card functionality
in the form of a higher level "wrapper" object.
License: GNU General Public License V3
(Because sdfatlib is licensed with this.)
(C) Copyright 2010 SparkFun Electronics
This library provides four key benefits:
* Including `SD.h` automatically creates a global
`SD` object which can be interacted with in a similar
manner to other standard global objects like `Serial` and `Ethernet`.
* Boilerplate initialisation code is contained in one method named
`begin` and no further objects need to be created in order to access
the SD card.
* Calls to `open` can supply a full path name including parent
directories which simplifies interacting with files in subdirectories.
* Utility methods are provided to determine whether a file exists
and to create a directory heirarchy.
Note however that not all functionality provided by the underlying
sdfatlib library is exposed.
*/
/*
Implementation Notes
In order to handle multi-directory path traversal, functionality that
requires this ability is implemented as callback functions.
Individual methods call the `walkPath` function which performs the actual
directory traversal (swapping between two different directory/file handles
along the way) and at each level calls the supplied callback function.
Some types of functionality will take an action at each level (e.g. exists
or make directory) which others will only take an action at the bottom
level (e.g. open).
*/
#include "SD.h"
// Used by `getNextPathComponent`
#define MAX_COMPONENT_LEN 12 // What is max length?
#define PATH_COMPONENT_BUFFER_LEN MAX_COMPONENT_LEN+1
bool getNextPathComponent(const char *path, unsigned int *p_offset,
char *buffer) {
/*
Parse individual path components from a path.
e.g. after repeated calls '/foo/bar/baz' will be split
into 'foo', 'bar', 'baz'.
This is similar to `strtok()` but copies the component into the
supplied buffer rather than modifying the original string.
`buffer` needs to be PATH_COMPONENT_BUFFER_LEN in size.
`p_offset` needs to point to an integer of the offset at
which the previous path component finished.
Returns `true` if more components remain.
Returns `false` if this is the last component.
(This means path ended with 'foo' or 'foo/'.)
*/
// TODO: Have buffer local to this function, so we know it's the
// correct length?
int bufferOffset = 0;
int offset = *p_offset;
// Skip root or other separator
if (path[offset] == '/') {
offset++;
}
// Copy the next next path segment
while (bufferOffset < MAX_COMPONENT_LEN
&& (path[offset] != '/')
&& (path[offset] != '\0')) {
buffer[bufferOffset++] = path[offset++];
}
buffer[bufferOffset] = '\0';
// Skip trailing separator so we can determine if this
// is the last component in the path or not.
if (path[offset] == '/') {
offset++;
}
*p_offset = offset;
return (path[offset] != '\0');
}
boolean walkPath(const char *filepath, SdFile& parentDir,
boolean (*callback)(SdFile& parentDir,
char *filePathComponent,
boolean isLastComponent,
void *object),
void *object = NULL) {
/*
When given a file path (and parent directory--normally root),
this function traverses the directories in the path and at each
level calls the supplied callback function while also providing
the supplied object for context if required.
e.g. given the path '/foo/bar/baz'
the callback would be called at the equivalent of
'/foo', '/foo/bar' and '/foo/bar/baz'.
The implementation swaps between two different directory/file
handles as it traverses the directories and does not use recursion
in an attempt to use memory efficiently.
If a callback wishes to stop the directory traversal it should
return false--in this case the function will stop the traversal,
tidy up and return false.
If a directory path doesn't exist at some point this function will
also return false and not subsequently call the callback.
If a directory path specified is complete, valid and the callback
did not indicate the traversal should be interrupted then this
function will return true.
*/
SdFile subfile1;
SdFile subfile2;
char buffer[PATH_COMPONENT_BUFFER_LEN];
unsigned int offset = 0;
SdFile *p_parent;
SdFile *p_child;
SdFile *p_tmp_sdfile;
p_child = &subfile1;
p_parent = &parentDir;
while (true) {
boolean moreComponents = getNextPathComponent(filepath, &offset, buffer);
boolean shouldContinue = callback((*p_parent), buffer, !moreComponents, object);
if (!shouldContinue) {
// TODO: Don't repeat this code?
// If it's one we've created then we
// don't need the parent handle anymore.
if (p_parent != &parentDir) {
(*p_parent).close();
}
return false;
}
if (!moreComponents) {
break;
}
boolean exists = (*p_child).open(*p_parent, buffer, O_RDONLY);
// If it's one we've created then we
// don't need the parent handle anymore.
if (p_parent != &parentDir) {
(*p_parent).close();
}
// Handle case when it doesn't exist and we can't continue...
if (exists) {
// We alternate between two file handles as we go down
// the path.
if (p_parent == &parentDir) {
p_parent = &subfile2;
}
p_tmp_sdfile = p_parent;
p_parent = p_child;
p_child = p_tmp_sdfile;
} else {
return false;
}
}
if (p_parent != &parentDir) {
(*p_parent).close(); // TODO: Return/ handle different?
}
return true;
}
/*
The callbacks used to implement various functionality follow.
Each callback is supplied with a parent directory handle,
character string with the name of the current file path component,
a flag indicating if this component is the last in the path and
a pointer to an arbitrary object used for context.
*/
boolean callback_pathExists(SdFile& parentDir, char *filePathComponent,
boolean isLastComponent, void *object) {
/*
Callback used to determine if a file/directory exists in parent
directory.
Returns true if file path exists.
*/
SdFile child;
(void) isLastComponent;
(void) object;
boolean exists = child.open(parentDir, filePathComponent, O_RDONLY);
if (exists) {
child.close();
}
return exists;
}
boolean callback_makeDirPath(SdFile& parentDir, char *filePathComponent,
boolean isLastComponent, void *object) {
/*
Callback used to create a directory in the parent directory if
it does not already exist.
Returns true if a directory was created or it already existed.
*/
boolean result = false;
SdFile child;
result = callback_pathExists(parentDir, filePathComponent, isLastComponent, object);
if (!result) {
result = child.makeDir(parentDir, filePathComponent);
}
return result;
}
/*
boolean callback_openPath(SdFile& parentDir, char *filePathComponent,
boolean isLastComponent, void *object) {
Callback used to open a file specified by a filepath that may
specify one or more directories above it.
Expects the context object to be an instance of `SDClass` and
will use the `file` property of the instance to open the requested
file/directory with the associated file open mode property.
Always returns true if the directory traversal hasn't reached the
bottom of the directory heirarchy.
Returns false once the file has been opened--to prevent the traversal
from descending further. (This may be unnecessary.)
if (isLastComponent) {
SDClass *p_SD = static_cast<SDClass*>(object);
p_SD->file.open(parentDir, filePathComponent, p_SD->fileOpenMode);
if (p_SD->fileOpenMode == FILE_WRITE) {
p_SD->file.seekSet(p_SD->file.fileSize());
}
// TODO: Return file open result?
return false;
}
return true;
}
*/
boolean callback_remove(SdFile& parentDir, char *filePathComponent,
boolean isLastComponent, void *object) {
(void) object;
if (isLastComponent) {
return SdFile::remove(parentDir, filePathComponent);
}
return true;
}
boolean callback_rmdir(SdFile& parentDir, char *filePathComponent,
boolean isLastComponent, void *object) {
(void) object;
if (isLastComponent) {
SdFile f;
if (!f.open(parentDir, filePathComponent, O_READ)) return false;
return f.rmDir();
}
return true;
}
/* Implementation of class used to create `SDCard` object. */
boolean SDClass::begin(uint8_t csPin, uint32_t speed) {
/*
Performs the initialisation required by the sdfatlib library.
Return true if initialization succeeds, false otherwise.
*/
return card.init(speed, csPin) &&
volume.init(card) &&
root.openRoot(volume);
}
//Warning: see comment in SD.h about possible card corruption.
void SDClass::end(bool endSPI)
{
if(card.errorCode() == 0 && root.isOpen()) {
root.close(); //Warning: this calls sync(), see above comment about corruption.
}
card.end(endSPI);
}
// this little helper is used to traverse paths
SdFile SDClass::getParentDir(const char *filepath, int *index) {
// get parent directory
SdFile d1 = root; // start with the mostparent, root!
SdFile d2;
// we'll use the pointers to swap between the two objects
SdFile *parent = &d1;
SdFile *subdir = &d2;
const char *origpath = filepath;
while (strchr(filepath, '/')) {
// get rid of leading /'s
if (filepath[0] == '/') {
filepath++;
continue;
}
if (! strchr(filepath, '/')) {
// it was in the root directory, so leave now
break;
}
// extract just the name of the next subdirectory
uint8_t idx = strchr(filepath, '/') - filepath;
if (idx > 12)
idx = 12; // dont let them specify long names
char subdirname[13];
strncpy(subdirname, filepath, idx);
subdirname[idx] = 0;
// close the subdir (we reuse them) if open
subdir->close();
if (! subdir->open(parent, subdirname, O_READ)) {
// failed to open one of the subdirectories
return SdFile();
}
// move forward to the next subdirectory
filepath += idx;
// we reuse the objects, close it.
parent->close();
// swap the pointers
SdFile *t = parent;
parent = subdir;
subdir = t;
}
*index = (int)(filepath - origpath);
// parent is now the parent diretory of the file!
return *parent;
}
File SDClass::open(const char *filepath, uint8_t mode) {
/*
Open the supplied file path for reading or writing.
The file content can be accessed via the `file` property of
the `SDClass` object--this property is currently
a standard `SdFile` object from `sdfatlib`.
Defaults to read only.
If `write` is true, default action (when `append` is true) is to
append data to the end of the file.
If `append` is false then the file will be truncated first.
If the file does not exist and it is opened for writing the file
will be created.
An attempt to open a file for reading that does not exist is an
error.
*/
int pathidx;
// do the interative search
SdFile parentdir = getParentDir(filepath, &pathidx);
// no more subdirs!
filepath += pathidx;
if (! filepath[0]) {
// it was the directory itself!
return File(parentdir, "/");
}
// Open the file itself
SdFile file;
// failed to open a subdir!
if (!parentdir.isOpen())
return File();
// there is a special case for the Root directory since its a static dir
if (parentdir.isRoot()) {
if ( ! file.open(root, filepath, mode)) {
// failed to open the file :(
return File();
}
// dont close the root!
} else {
if ( ! file.open(parentdir, filepath, mode)) {
return File();
}
// close the parent
parentdir.close();
}
if (mode & (O_APPEND | O_WRITE))
file.seekSet(file.fileSize());
return File(file, filepath);
}
/*
File SDClass::open(char *filepath, uint8_t mode) {
//
Open the supplied file path for reading or writing.
The file content can be accessed via the `file` property of
the `SDClass` object--this property is currently
a standard `SdFile` object from `sdfatlib`.
Defaults to read only.
If `write` is true, default action (when `append` is true) is to
append data to the end of the file.
If `append` is false then the file will be truncated first.
If the file does not exist and it is opened for writing the file
will be created.
An attempt to open a file for reading that does not exist is an
error.
//
// TODO: Allow for read&write? (Possibly not, as it requires seek.)
fileOpenMode = mode;
walkPath(filepath, root, callback_openPath, this);
return File();
}
*/
//boolean SDClass::close() {
// /*
//
// Closes the file opened by the `open` method.
//
// */
// file.close();
//}
boolean SDClass::exists(const char *filepath) {
/*
Returns true if the supplied file path exists.
*/
return walkPath(filepath, root, callback_pathExists);
}
//boolean SDClass::exists(char *filepath, SdFile& parentDir) {
// /*
//
// Returns true if the supplied file path rooted at `parentDir`
// exists.
//
// */
// return walkPath(filepath, parentDir, callback_pathExists);
//}
boolean SDClass::mkdir(const char *filepath) {
/*
Makes a single directory or a heirarchy of directories.
A rough equivalent to `mkdir -p`.
*/
return walkPath(filepath, root, callback_makeDirPath);
}
boolean SDClass::rmdir(const char *filepath) {
/*
Remove a single directory or a heirarchy of directories.
A rough equivalent to `rm -rf`.
*/
return walkPath(filepath, root, callback_rmdir);
}
boolean SDClass::remove(const char *filepath) {
return walkPath(filepath, root, callback_remove);
}
// allows you to recurse into a directory
File File::openNextFile(uint8_t mode) {
dir_t p;
//Serial.print("\t\treading dir...");
while (_file->readDir(&p) > 0) {
// done if past last used entry
if (p.name[0] == DIR_NAME_FREE) {
//Serial.println("end");
return File();
}
// skip deleted entry and entries for . and ..
if (p.name[0] == DIR_NAME_DELETED || p.name[0] == '.') {
//Serial.println("dots");
continue;
}
// only list subdirectories and files
if (!DIR_IS_FILE_OR_SUBDIR(&p)) {
//Serial.println("notafile");
continue;
}
// print file name with possible blank fill
SdFile f;
char name[13];
_file->dirName(p, name);
//Serial.print("try to open file ");
//Serial.println(name);
if (f.open(_file, name, mode)) {
//Serial.println("OK!");
return File(f, name);
} else {
//Serial.println("ugh");
return File();
}
}
//Serial.println("nothing");
return File();
}
void File::rewindDirectory(void) {
if (isDirectory())
_file->rewind();
}
#if !defined(NO_GLOBAL_INSTANCES) && !defined(NO_GLOBAL_SD)
SDClass SD;
#endif