* Update to upstream SdFat 2.0.2
Increases the read/write performance for SD card accesses
by a significant amount, up to 5x (3+MB/s) in testing.
Fixes#7772
* Add SDFS::availableForWrite handler
Peek into the sector cache to determine the maximum number of
bytes that can be written w/o needing a (slow) SD operation.
Fixes#7650
The SDFS implementation didn't include plumbing to support user-supplied
timestamp generators (for file create/write times) because the SDFat
library doesn't support a callback parameter.
Work around it by using a single, global (inside sdfs namespace) that
the static timestamp callback member can see and use.
Add a test of the feature in CI.
Fixes#7682
* Add Print::availableForWrite method
Adds an availableForWrite() method to the Print class, matching current
ArduinoCore-API commit 398e70f188e2b861c10d9ffe5e2bfcb6a4a4f489 .
Hook availableForWrite into the SDFS filesystem (other FSes don't have
this capability built-in).
Fixes#7650
* WiFiClient::availableForWrite proto matching Print
* Fix Netdump signedness warning
* Clean up Serial availableForWrite
This is evidently a breaking change due to the type difference.
Arduino's `availableForWrite` returns an `int`, while the
(multiply-implemented, non-virtual) core `availableForWrite` returned
`size_t`.
* Add fileCreation/getCreation create-time accessors
For SDFS and LittleFS, enable a creation time accessor for files and Dir
iterators, similar to the existing fileTime/getLastWrite calls.
Remove spurious Dir::getLastWrite method (the proper and only documented
way is really Dir::fileTime).
Update json to point to new mklittlefs which copies the creation date of
files to the image.
Fixes#6992
* Remove malloc(), use stack vars for temp names
LFS filenames are limited in size and generally very small. Use a stack
variable instead of a dynamic allocation when performing full-path
computations.
* Replace "Creation" w/"CreationTime" in FS accessor
Per review, `getCreation` -> `getCreationTime`, `fileCreation` ->
`fileCreationTime`.
The names `fileTime()` and `getLastWrite()` are inherited from ESP32
implementation and unchanged.
* Add creation time to listfiles SD example
* Enable SdFat's sateTime callback for timestamping
SdFat requries the dateTimeCallback call (global for everything) to
update dates and times on created files.
Because the callback signature doesn't have space for us to provide
any parameters, we cannot get the the File, Dir, or FS object's
dateTimeCB member. Instead, just go with `time(null)` as the callback
function which is right in all but the most esoteric cases.
* Correct DOS year/month offset in dateTime callback
* Fix docs to match new xxxCreationTime() API names
Co-authored-by: Develo <deveyes@gmail.com>
Forgot to add a Dir->fileTime override, resulting in it always returning
(time_t)0, or Jan 1, 1970.
Add the override, returning the proper lastWriteTime.
Fixes#6981
* Add time to filesystem API
Support the ESP32 File::getLastWrite() call and setting the time on
all filesystems automatically (assuming the system clock has
been set properly and time(NULL) returns the proper time!).
Adds Dir::fileTime() to get the time of a file being listed, similar to
Dir::fileName() and Dir::fileSize().
Adds ::setTimeCallback(time_t (*cb)()) to File, Dir, and FS, allowing
users to override the default timestamp on a per-file, directory, or
filesystem basis. By default, a simple callback returning time(nullptr)
is implemented.
LittleFS uses the 't' attribute and should be backwards compatible.
SD/SDFS work and include wrappers for obsolete SdFat timestamp callbacks
using the MSDOS time.
This PR does not update SPIFFS, due to compatability concerns and a
possible massive rewrite which would make it possible to determine if an
old-style ot metadata enabled FS is present at mount time.
Includes an updated SD/listfiles and LittleFS_time example.
Replaces #6315
* Add links to new mklittlefs w/timestamp support
Include the update mklittlefs which generated 't' metadata on imported
files.
../tools/sdk/lwip2/include/netif/lowpan6_opts.h
* Add explicit note about timestamp being local time
* Address review concerns
Clean up some awkward object instantiations.
Remove the _enableTime flag/setter from SPIFFS.
Clean up the FSConfig constructors using C++ style init lists.
Fixes#6082
Add an info64() call which returns used and total sizes as 64 bit
quantities. A default wrapper that just copies the 32-bit values is
included for LittleFS/SPIFFS which can't hit those capacities.
Fixes#6081
The SD rewrite blanked out some of the internal FAT info.. Restore the
function calls and return proper values.
Because size() is used in many printf()s, we can't just change its
return type to uint64. Instead, when size is > size-max warn.
Add SD.size64 which can be used by new apps who care about >4GB cards.
Prints a warning if debugging enabled
* polledTimeout: add option to use CPU count instead of millis()
* use more "using" alias
* more c++/clear code, using typename (thanks @devyte)
* rename class name to include unit, introduce timeMax() and check it with assert()
* remove useless defines
* improve api readability, add micro-second unit
* update example
* mock: emulate getCycleCount, add/fix polledTimeout CI test
* + nano-seconds, assert -> message, comments, host test
* allow 0 for timeout (enables immediate timeout, fix division by 0)
* typo, set member instead of local variable
* unify error message
* slight change on checkExpired() allows "never expired"
also removed printed message, add YieldAndDelay, simplify calculations
* remove traces of debug.h/cpp in this PR
* include missing <limits> header
* back to original expired test, introduce boolean _neverExpires, fix reset(), getTimeout() is invalid
* fix expiredOneShot with _timeout==0 check
* reenable getTimeout()
* expose checkExpired with unit conversion
* fix timing comments, move critical code to iram
* add member ::neverExpires and use it where relevant
* improve clarity
* remove exposed checkExpired(), adapt LEAmDNS with equivalent
* add API ::resetToNeverExpires(), use it in LEAmDNS
* remove offending constness from ::flagged() LEAmDNS (due do API fix in PolledTimeout)
* simplify "Fast" base classes
* minor variable rename
* Fix examples
* compliance with good c++ manners
* minor changes for consistency
* add missing const
* expired() and bool() moved to iram
* constexpr compensation computing
* add/update comments
* move neverExpires and alwaysExpired
* Add a FAT filesystem for SD cards to Arduino FS
Arduino forked a copy of SD lib several years ago, put their own wrapper
around it, and it's been languishing in our ESP8266 libraries ever since
as SD. It doesn't support long file names, has class names which
conflict with the ESP8266 internal names, and hasn't been updated in
ages.
The original author of the SD library has continued work in the
meantime, and produced a very feature rich implementation of SdFat. It
unfortunately also conflicts with the class names we use in ESP8266
Arduino and has a different API than the internal SPIFFS or proposed
LittleFS filesystem objects.
This PR puts a wrapper around the latest and greatest SdFat library,
by forking it and wrapping its classes in a private namespace "sdfat,"
and making as thin a wrapper as possible around it to conform to
the ESP8266 FS, File, and Dir classes.
This PR also removes the Arduino SD.h class library and rewrites it
using the new SDFS filesystem to make everything in the ESP8266
Arduino core compatible with each other.
By doing so it lets us use a single interface for anything needing a
file instead of multiple ones (see SDWebServer and how a different
object is needed vs. one serving from SPIFFS even though the logic is
all the same). Same for BearSSL's CertStores and probably a few others
I've missed, cleaning up our code base significantly.
Like LittleFS, silently create directories when a file is created with
a subdirectory specifier ("/path/to/file.txt") if they do not yet exist.
Adds a blacklist of sketches to skip in the CI process (because SdFat
has many examples which do not build properly on the ESP8266).
Now that LittleFS and SDFS have directory support, the FS needs to be
able to communicate whether a name is one or the other. Add a simple
bool FS::isDirectory() and bool FS::isFile() method. SPIFFS doesn't
have directories, so if it's valid it's a file and reported as such.
Add ::mkdir/::rmdir to the FS class to allow users to make and destroy
subdirectories. SPIFFS directory operations will, of course, fail
and return false.
Emulate a 16MB SD card and allow test runner to exercise it by using
a custom SdFat HOST_MOCK-enabled object.
Throw out the original Arduino SD.h class and rewrite from scratch using
only the ESP8266 native SDFS calls. This makes "SD" based applications
compatible with normal ESP8266 "File" and "FS" and "SPIFFS" operations.
The only major visible change for users is that long filenames now are
fully supported and work without any code changes. If there are static
arrays of 11 bytes for old 8.3 names in code, they will need to be
adjusted.
While it is recommended to use the more powerful SDFS class to access SD
cards, this SD.h wrapper allows for use of existing Arduino libraries
which are built to only with with that SD class.
Additional helper functions added to ESP8266 native Filesystem:: classes
to help support this portability.
The rewrite is good enough to run the original SDWebServer and SD
example code without any changes.
* Add a FSConfig and SDFSConfig param to FS.begin()
Allows for configuration values to be passed into a filesystem via the
begin method. By default, a FS will receive a nullptr and should so
whatever is appropriate.
The base FSConfig class has one parameter, _autoFormat, set by the
default constructor to true.
For SPIFFS, you can now disable auto formatting on mount failure by
passing in a FSConfig(false) object.
For SDFS a SDFSConfig parameter can be passed into config specifying the
chip select and SPI configuration. If nothing is passed in, the begin
will fail since there are no safe default values here.
* Add FS::setConfig to set FS-specific options
Add a new call, FS::setConfig(const {SDFS,SPIFFS}Config *cfg), which
takes a FS-specific configuration object and copies any special settings
on a per-FS basis. The call is only valid on unmounted filesystems, and
checks the type of object passed in matches the FS being configured.
Updates the docs and tests to utilize this new configuration method.
* Add ::truncate to File interface
Fixes#3846
* Use polledTimeout for formatting yields, cleanup
Use the new polledTimeout class to ensure a yield every 5ms while
formatting.
Add in default case handling and some debug messages when invalid inputs
specified.
* Make setConfig take const& ref, cleaner code
setConfig now can take a parameter defined directly in the call by using
a const &ref to it, leading to one less line of code to write and
cleaner reading of the code.
Also clean up SDFS implementation pointer definition.
