Stop CI from pulling LLVM repos and using GNUPG keyservers,
ubuntu-latest already has clang-format-{13,14,15}
Fixes long-standing issue with -style=file:...
* Remove warnings, errors during host tests in CI
Debug strings often included format parameters which did not exactly match
the passed in format parameters, resulting in warnings in the host test build
process like
````
/home/runner/work/Arduino/Arduino/libraries/ESP8266WiFi/src/ESP8266WiFiAP.cpp:107:20: warning: format ‘%u’ expects argument of type ‘unsigned int’, but argument 3 has type ‘size_t’ {aka ‘long unsigned int’} [-Wformat=]
107 | DEBUG_WIFI("[AP] SSID length %u, too long or missing!\n", ssid_len);
| ~~~~~~~~
| |
| size_t {aka long unsigned int}
````
Fix by applying casting or PRxxx macros as appropriate.
Also, fix one debug message which was trying to use a `String` as a `char *`:
````
/home/runner/work/Arduino/Arduino/libraries/ESP8266WiFi/src/ESP8266WiFiMulti.cpp: In member function ‘wl_status_t ESP8266WiFiMulti::connectWiFiMulti(uint32_t)’:
/home/runner/work/Arduino/Arduino/libraries/ESP8266WiFi/src/ESP8266WiFiMulti.cpp:331:34: warning: format ‘%s’ expects argument of type ‘char*’, but argument 3 has type ‘String’ [-Wformat=]
331 | DEBUG_WIFI_MULTI("[WIFIM] Connecting %s\n", ssid);
````
* Clean up SpeedTest.ino host build
Use a formatting function to give more human-readable output formats
for flash bandwidth. When the test starts and ends in less than one
millisecond, report as "Infinite".
Sample output:
````
Creating 512KB file, may take a while...
==> Time to write 512KB in 256b chunks = 6641 milliseconds
==> Created file size = 524288
Reading 512KB file sequentially in 256b chunks
==> Time to read 512KB sequentially in 256b chunks = 211 milliseconds = 2.48 MB/s
Reading 512KB file MISALIGNED in flash and RAM sequentially in 256b chunks
==> Time to read 512KB sequentially MISALIGNED in flash and RAM in 256b chunks = 212 milliseconds = 2.47 MB/s
Reading 512KB file in reverse by 256b chunks
==> Time to read 512KB in reverse in 256b chunks = 367 milliseconds = 1.43 MB/s
Writing 64K file in 1-byte chunks
==> Time to write 64KB in 1b chunks = 1249 milliseconds = 52.47 KB/s
Reading 64K file in 1-byte chunks
==> Time to read 64KB in 1b chunks = 296 milliseconds = 221.41 KB/s
````
When LittleFS.begin() or SDFS.begin() is called after the filesystem is
already mounted, don't unmount/remount. When an unmount happens, all old
Files become invalid (but the core doesn't know this), so you would end
up with random crashes in FS code.
Now, check for _mounted, and if so just return immediately from begin().
This mimics the original SPIFFS code.
Fixes https://github.com/earlephilhower/ESP8266Audio/issues/407
* Don't crash when includeing LittleFS.h w/no FS
The LittleFS constructor could cause a divide-by-zero error and crash the
chip during pre-main startup (i.e. when the constructors were called).
Avoid by only initializing the LittleFS control structure when there is
a filesystem specified in the flash layout.
* Be even more paranoid on begin() and format()
* Upgrade to upstream newlib 4.0.0 release
Includes 64 bit time_t and 5 years of updates.
Binary incompatible with libraries which use time_t (due to the size
difference). Recompiling with the new newlib should be sufficient for
most libraries, assuming source is available.
* Remove tools/sdk/libc directory, it isn't used anywhere
Somewhere along the line the copy of libc in tools/sdl/libc was taken
out of the build process. Files in there are not used, take add'l time
to build and install on a toolchain release, and just cause confusion.
Remove them.
* Fix 64-bit time for LittleFS
The core was setting 64-bit times automatically on new file creation or
updates, but would fail when attempting to read them back due to 64/32b
confusion.
Now attempt to read 64b time, and if that fails fallback to reading 32b
time to allow both old and new FS to preserve timestamps.
* Update to jjsuwa-sys3175 additions to GCC and newlib
@jjsuwa-sys3175 contributed multiple patches to GCC, included in
the toolchain, as well as a slightly faster pgm_read_byte() macro.
* Rebuild w/addl GCC patches, new BearSSL flags
* Remove copied libgcc.a file, is contained in toolchain
Fixes#7775
Clean up the passing/setting of custom File time callbacks and add a
host test verifying they work. Existing core was not passing custom
timeCallbacks set at the FS level down to open()ed files, resulting in
them calling the default time(nullptr) and reporting wrong file modify
times.
* Free space of overwritten files in LittleFS
Fixes#7426
LittleFS doesn't update the on-flash data structures when a file is
reopened as O_TRUNC until the file is closed. This means the space of
the original, inaccessible file cannot be used, causing OOS errors in
cases when a large file is being overwritten.
Explicitly call the file sync operation to update the on-flash metadata
as soon as a file is opened. For normal files it's a no-op, but for
O_TRUNC modes it will free the space, allowing full overwrite of large
files.
* Add host test case for change
Fixes#7323
While I'm not a fan, the Arduino FileSeek API online shows that a seek()
past EOF should return FALSE.
https://www.arduino.cc/en/Reference/FileSeek
SPIFFS and SDFS obey this, but LittleFS followed the POSIX standard or
allowing seeks past EOF.
Update LittleFS::seek() to follow the Arduino API and add tests for it.
* Deprecate SPIFFS, move examples to LittleFS
SPIFFS has been a great filesystem, but it has significant problems in
many cases (and it's also pretty slow). Development seems to have
slowed/stopped on the upstream version, and we're not able to provide
support or fix the known issues with it as-is.
Deprecate SPIFFS variable.
Update all examples to use LittleFS instead of SPIFFS.
Also, minor cleanup on very old examples which has obsolete delays
waiting for the Serial port to come up, or which were stuck at 9600 baud
because of their ancient AVR heritage.
Fixes#7095
* Remove leftover debug code
* Clean up comments in some examples
* Update documentation on SPIFFS deprecation
* Fix host tests to avoid deprecation warnings
* Fix cut-n-paste error
* Restore SpeedTest.ino, adjust to allow custom FSes
Co-authored-by: Develo <deveyes@gmail.com>
* Add fileCreation/getCreation create-time accessors
For SDFS and LittleFS, enable a creation time accessor for files and Dir
iterators, similar to the existing fileTime/getLastWrite calls.
Remove spurious Dir::getLastWrite method (the proper and only documented
way is really Dir::fileTime).
Update json to point to new mklittlefs which copies the creation date of
files to the image.
Fixes#6992
* Remove malloc(), use stack vars for temp names
LFS filenames are limited in size and generally very small. Use a stack
variable instead of a dynamic allocation when performing full-path
computations.
* Replace "Creation" w/"CreationTime" in FS accessor
Per review, `getCreation` -> `getCreationTime`, `fileCreation` ->
`fileCreationTime`.
The names `fileTime()` and `getLastWrite()` are inherited from ESP32
implementation and unchanged.
* Add creation time to listfiles SD example
* Enable SdFat's sateTime callback for timestamping
SdFat requries the dateTimeCallback call (global for everything) to
update dates and times on created files.
Because the callback signature doesn't have space for us to provide
any parameters, we cannot get the the File, Dir, or FS object's
dateTimeCB member. Instead, just go with `time(null)` as the callback
function which is right in all but the most esoteric cases.
* Correct DOS year/month offset in dateTime callback
* Fix docs to match new xxxCreationTime() API names
Co-authored-by: Develo <deveyes@gmail.com>
To mimic SPIFFs behavior, we automatically create subdirectories when a
file is opened in a subdir.
The check mixed up a bitmask check with a boolean AND. Fix it.
Fixes#6955
LittleFS was updating the timestamp on any close, not only for files
when they were opened for writing. This could lead to excessive writes
to the flash.
Preserve the LFS flags, and only update the timestamp if the file was
opened for writing.
* define two weak functions defaulting to no-op
redefine them to do something useful when either spiffs or littlefs are used
* noop
* single entry point for closing FSes
* rename functions, override when instanciated, add link to explanation
* spiffs: call end on destructor
* Add time to filesystem API
Support the ESP32 File::getLastWrite() call and setting the time on
all filesystems automatically (assuming the system clock has
been set properly and time(NULL) returns the proper time!).
Adds Dir::fileTime() to get the time of a file being listed, similar to
Dir::fileName() and Dir::fileSize().
Adds ::setTimeCallback(time_t (*cb)()) to File, Dir, and FS, allowing
users to override the default timestamp on a per-file, directory, or
filesystem basis. By default, a simple callback returning time(nullptr)
is implemented.
LittleFS uses the 't' attribute and should be backwards compatible.
SD/SDFS work and include wrappers for obsolete SdFat timestamp callbacks
using the MSDOS time.
This PR does not update SPIFFS, due to compatability concerns and a
possible massive rewrite which would make it possible to determine if an
old-style ot metadata enabled FS is present at mount time.
Includes an updated SD/listfiles and LittleFS_time example.
Replaces #6315
* Add links to new mklittlefs w/timestamp support
Include the update mklittlefs which generated 't' metadata on imported
files.
../tools/sdk/lwip2/include/netif/lowpan6_opts.h
* Add explicit note about timestamp being local time
* Address review concerns
Clean up some awkward object instantiations.
Remove the _enableTime flag/setter from SPIFFS.
Clean up the FSConfig constructors using C++ style init lists.
* Update LittleFS to current project head
Several bugfixes noted in the LittleFS added:
https://github.com/ARMmbed/littlefs/commits/master
* Also update SdFat to latest upstream head
Also adds a delay(0) every 5000 fat clusters examined to avoid WDT
errors on highly fragmented FAT filesystems.
* enable by default latest 2.2.x firmware, including fixed espnow
* LittleFS: avoid crash when FS size is 0
* flash size defaults: 1M for generic board, not empty FS for all
Fixes#6082
Add an info64() call which returns used and total sizes as 64 bit
quantities. A default wrapper that just copies the 32-bit values is
included for LittleFS/SPIFFS which can't hit those capacities.
* Add LittleFS as internal flash filesystem
Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem,
https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard
flash utilizing the exact same API as the existing SPIFFS filesystem.
LittleFS is built for low memory systems that are subject to random power
losses, is actively supported by the ARMmbed community, supports directories,
and seems to be much faster in the large-ish read-mostly applications I use.
LittleFS, however, has a larger minimum file allocation unit and does not do
static wear levelling. This means that for systems that need many little
files (<4K), have small SPIFFS areas (64K), or which have a large static
set of files covering the majority of flash coupled with a frequently
updated set of other files, it may not perform as well.
Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch,
use LittleFS.open in place of SPIFFS.open to open files, and everything
else just works thanks to the magic of @igrr's File base class.
**LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS**
Since it is a completely different filesystem, you will need to reformat
your flash (and lose any data therein) to use it. Tools to build the
flash filesystem and upload are at
https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and
https://github.com/earlephilhower/mklittlefs/ . The mklittlefs tool
is installed as part of the Arduino platform installation, automatically.
The included example shows a contrived read-mostly example and
demonstrates how the same calls work on either SPIFFS.* or LittleFS.*
Host tests are also included as part of CI.
Directories are fully supported in LittleFS. This means that LittleFS
will have a slight difference vs. SPIFFS when you use
LittleFS.openDir()/Dir.next(). On SPIFFS dir.next()
will return all filesystem entries, including ones in "subdirs"
(because in SPIFFS there are no subdirs and "/" is the same as any
other character in a filename).
On LittleFS, dir.next() will only return entries in the directory
specified, not subdirs. So to list files in "/subdir/..." you need
to actually openDir("/subdir") and use Dir.next() to parse through
just those elements. The returned filenames also only have the
filename returned, not full paths. So on a FS with "/a/1", "/a/2"
when you do openDir("/a"); dir.next().getName(); you get "1" and "2"
and not "/a/1" and "/a/2" like in SPIFFS. This is consistent with
POSIX ideas about reading directories and more natural for a FS.
Most code will not be affected by this, but if you depend on
openDir/Dir.next() you need to be aware of it.
Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile,
::openNextFile, and ::rewind methods added to Filesystem objects.
Documentation has been updated with this and other LittleFS information.
Subdirectories are made silently when they do not exist when you
try and create a file in a subdir. They are silently removed when
the last file in them is deleted. This is consistent with what
SPIFFS does but is obviously not normal POSIX behavior. Since there
has never been a "FS.mkdir()" method this is the only way to be
compatible with legacy SPIFFS code.
SPIFFS code has been refactored to pull out common flash_hal_* ops
and placed in its own namespace, like LittleFS.
* Fix up merge blank line issue
* Merge in the FSConfig changs from SDFS PR
Enable setConfig for LittleFS as well plys merge the SPIFFS changes
done in the SDFS PR.
* Fix merge errors
* Update to use v2-alpha branch
The V2-alpha branch supports small file optimizations which can help
increase the utilization of flash when small files are prevalent.
It also adds support for metadata, which means we can start adding
things like file creation times, if desired (not yet).
* V2 of littlefs is now in upstream/master
* Update test to support non-creation-ordered files
In a directory, the order in which "readNextFile()" will return a name
is undefined. SPIFFS may return it in order, but LittleFS does not as
of V2. Update the test to look for files by name when doing
readNextFile() testing.
* Fix LittleFS.truncate implementation
* Fix SDFS tests
SDFS, SPIFFS, and LittleFS now all share the same common set of tests,
greatly increasing the SDFS test coverage.
* Update to point to mklittlefs v2
Upgrade mklittlefs to V2 format support
* Remove extra FS::write(const char *s) method
This was removed in #5861 and erroneously re-introduced here.
* Minimize spurious differences from master
* Dramatically reduce memory usage
Reduce the program and read chunk sizes which impacts performance
minimally but reduces per-file RAM usage of 16KB to <1KB.
* Add @d-a-v's host emulation for LittleFS
* Fix SW Serial library version
* Fix free space reporting
Thanks to @TD-er for discovering the issue
* Update littlefs to latest upstream
* Remove sdfat version included by accident
* Update SDFAT to include MOCK changes required
* Update to include SD.h test of file append
