* Deprecate SPIFFS, move examples to LittleFS
SPIFFS has been a great filesystem, but it has significant problems in
many cases (and it's also pretty slow). Development seems to have
slowed/stopped on the upstream version, and we're not able to provide
support or fix the known issues with it as-is.
Deprecate SPIFFS variable.
Update all examples to use LittleFS instead of SPIFFS.
Also, minor cleanup on very old examples which has obsolete delays
waiting for the Serial port to come up, or which were stuck at 9600 baud
because of their ancient AVR heritage.
Fixes#7095
* Remove leftover debug code
* Clean up comments in some examples
* Update documentation on SPIFFS deprecation
* Fix host tests to avoid deprecation warnings
* Fix cut-n-paste error
* Restore SpeedTest.ino, adjust to allow custom FSes
Co-authored-by: Develo <deveyes@gmail.com>
* Add fileCreation/getCreation create-time accessors
For SDFS and LittleFS, enable a creation time accessor for files and Dir
iterators, similar to the existing fileTime/getLastWrite calls.
Remove spurious Dir::getLastWrite method (the proper and only documented
way is really Dir::fileTime).
Update json to point to new mklittlefs which copies the creation date of
files to the image.
Fixes#6992
* Remove malloc(), use stack vars for temp names
LFS filenames are limited in size and generally very small. Use a stack
variable instead of a dynamic allocation when performing full-path
computations.
* Replace "Creation" w/"CreationTime" in FS accessor
Per review, `getCreation` -> `getCreationTime`, `fileCreation` ->
`fileCreationTime`.
The names `fileTime()` and `getLastWrite()` are inherited from ESP32
implementation and unchanged.
* Add creation time to listfiles SD example
* Enable SdFat's sateTime callback for timestamping
SdFat requries the dateTimeCallback call (global for everything) to
update dates and times on created files.
Because the callback signature doesn't have space for us to provide
any parameters, we cannot get the the File, Dir, or FS object's
dateTimeCB member. Instead, just go with `time(null)` as the callback
function which is right in all but the most esoteric cases.
* Correct DOS year/month offset in dateTime callback
* Fix docs to match new xxxCreationTime() API names
Co-authored-by: Develo <deveyes@gmail.com>
* Add time to filesystem API
Support the ESP32 File::getLastWrite() call and setting the time on
all filesystems automatically (assuming the system clock has
been set properly and time(NULL) returns the proper time!).
Adds Dir::fileTime() to get the time of a file being listed, similar to
Dir::fileName() and Dir::fileSize().
Adds ::setTimeCallback(time_t (*cb)()) to File, Dir, and FS, allowing
users to override the default timestamp on a per-file, directory, or
filesystem basis. By default, a simple callback returning time(nullptr)
is implemented.
LittleFS uses the 't' attribute and should be backwards compatible.
SD/SDFS work and include wrappers for obsolete SdFat timestamp callbacks
using the MSDOS time.
This PR does not update SPIFFS, due to compatability concerns and a
possible massive rewrite which would make it possible to determine if an
old-style ot metadata enabled FS is present at mount time.
Includes an updated SD/listfiles and LittleFS_time example.
Replaces #6315
* Add links to new mklittlefs w/timestamp support
Include the update mklittlefs which generated 't' metadata on imported
files.
../tools/sdk/lwip2/include/netif/lowpan6_opts.h
* Add explicit note about timestamp being local time
* Address review concerns
Clean up some awkward object instantiations.
Remove the _enableTime flag/setter from SPIFFS.
Clean up the FSConfig constructors using C++ style init lists.
* Add LittleFS as internal flash filesystem
Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem,
https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard
flash utilizing the exact same API as the existing SPIFFS filesystem.
LittleFS is built for low memory systems that are subject to random power
losses, is actively supported by the ARMmbed community, supports directories,
and seems to be much faster in the large-ish read-mostly applications I use.
LittleFS, however, has a larger minimum file allocation unit and does not do
static wear levelling. This means that for systems that need many little
files (<4K), have small SPIFFS areas (64K), or which have a large static
set of files covering the majority of flash coupled with a frequently
updated set of other files, it may not perform as well.
Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch,
use LittleFS.open in place of SPIFFS.open to open files, and everything
else just works thanks to the magic of @igrr's File base class.
**LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS**
Since it is a completely different filesystem, you will need to reformat
your flash (and lose any data therein) to use it. Tools to build the
flash filesystem and upload are at
https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and
https://github.com/earlephilhower/mklittlefs/ . The mklittlefs tool
is installed as part of the Arduino platform installation, automatically.
The included example shows a contrived read-mostly example and
demonstrates how the same calls work on either SPIFFS.* or LittleFS.*
Host tests are also included as part of CI.
Directories are fully supported in LittleFS. This means that LittleFS
will have a slight difference vs. SPIFFS when you use
LittleFS.openDir()/Dir.next(). On SPIFFS dir.next()
will return all filesystem entries, including ones in "subdirs"
(because in SPIFFS there are no subdirs and "/" is the same as any
other character in a filename).
On LittleFS, dir.next() will only return entries in the directory
specified, not subdirs. So to list files in "/subdir/..." you need
to actually openDir("/subdir") and use Dir.next() to parse through
just those elements. The returned filenames also only have the
filename returned, not full paths. So on a FS with "/a/1", "/a/2"
when you do openDir("/a"); dir.next().getName(); you get "1" and "2"
and not "/a/1" and "/a/2" like in SPIFFS. This is consistent with
POSIX ideas about reading directories and more natural for a FS.
Most code will not be affected by this, but if you depend on
openDir/Dir.next() you need to be aware of it.
Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile,
::openNextFile, and ::rewind methods added to Filesystem objects.
Documentation has been updated with this and other LittleFS information.
Subdirectories are made silently when they do not exist when you
try and create a file in a subdir. They are silently removed when
the last file in them is deleted. This is consistent with what
SPIFFS does but is obviously not normal POSIX behavior. Since there
has never been a "FS.mkdir()" method this is the only way to be
compatible with legacy SPIFFS code.
SPIFFS code has been refactored to pull out common flash_hal_* ops
and placed in its own namespace, like LittleFS.
* Fix up merge blank line issue
* Merge in the FSConfig changs from SDFS PR
Enable setConfig for LittleFS as well plys merge the SPIFFS changes
done in the SDFS PR.
* Fix merge errors
* Update to use v2-alpha branch
The V2-alpha branch supports small file optimizations which can help
increase the utilization of flash when small files are prevalent.
It also adds support for metadata, which means we can start adding
things like file creation times, if desired (not yet).
* V2 of littlefs is now in upstream/master
* Update test to support non-creation-ordered files
In a directory, the order in which "readNextFile()" will return a name
is undefined. SPIFFS may return it in order, but LittleFS does not as
of V2. Update the test to look for files by name when doing
readNextFile() testing.
* Fix LittleFS.truncate implementation
* Fix SDFS tests
SDFS, SPIFFS, and LittleFS now all share the same common set of tests,
greatly increasing the SDFS test coverage.
* Update to point to mklittlefs v2
Upgrade mklittlefs to V2 format support
* Remove extra FS::write(const char *s) method
This was removed in #5861 and erroneously re-introduced here.
* Minimize spurious differences from master
* Dramatically reduce memory usage
Reduce the program and read chunk sizes which impacts performance
minimally but reduces per-file RAM usage of 16KB to <1KB.
* Add @d-a-v's host emulation for LittleFS
* Fix SW Serial library version
* Fix free space reporting
Thanks to @TD-er for discovering the issue
* Update littlefs to latest upstream
* Remove sdfat version included by accident
* Update SDFAT to include MOCK changes required
* Update to include SD.h test of file append
Fixes#6081
The SD rewrite blanked out some of the internal FAT info.. Restore the
function calls and return proper values.
Because size() is used in many printf()s, we can't just change its
return type to uint64. Instead, when size is > size-max warn.
Add SD.size64 which can be used by new apps who care about >4GB cards.
Prints a warning if debugging enabled
* Add a FAT filesystem for SD cards to Arduino FS
Arduino forked a copy of SD lib several years ago, put their own wrapper
around it, and it's been languishing in our ESP8266 libraries ever since
as SD. It doesn't support long file names, has class names which
conflict with the ESP8266 internal names, and hasn't been updated in
ages.
The original author of the SD library has continued work in the
meantime, and produced a very feature rich implementation of SdFat. It
unfortunately also conflicts with the class names we use in ESP8266
Arduino and has a different API than the internal SPIFFS or proposed
LittleFS filesystem objects.
This PR puts a wrapper around the latest and greatest SdFat library,
by forking it and wrapping its classes in a private namespace "sdfat,"
and making as thin a wrapper as possible around it to conform to
the ESP8266 FS, File, and Dir classes.
This PR also removes the Arduino SD.h class library and rewrites it
using the new SDFS filesystem to make everything in the ESP8266
Arduino core compatible with each other.
By doing so it lets us use a single interface for anything needing a
file instead of multiple ones (see SDWebServer and how a different
object is needed vs. one serving from SPIFFS even though the logic is
all the same). Same for BearSSL's CertStores and probably a few others
I've missed, cleaning up our code base significantly.
Like LittleFS, silently create directories when a file is created with
a subdirectory specifier ("/path/to/file.txt") if they do not yet exist.
Adds a blacklist of sketches to skip in the CI process (because SdFat
has many examples which do not build properly on the ESP8266).
Now that LittleFS and SDFS have directory support, the FS needs to be
able to communicate whether a name is one or the other. Add a simple
bool FS::isDirectory() and bool FS::isFile() method. SPIFFS doesn't
have directories, so if it's valid it's a file and reported as such.
Add ::mkdir/::rmdir to the FS class to allow users to make and destroy
subdirectories. SPIFFS directory operations will, of course, fail
and return false.
Emulate a 16MB SD card and allow test runner to exercise it by using
a custom SdFat HOST_MOCK-enabled object.
Throw out the original Arduino SD.h class and rewrite from scratch using
only the ESP8266 native SDFS calls. This makes "SD" based applications
compatible with normal ESP8266 "File" and "FS" and "SPIFFS" operations.
The only major visible change for users is that long filenames now are
fully supported and work without any code changes. If there are static
arrays of 11 bytes for old 8.3 names in code, they will need to be
adjusted.
While it is recommended to use the more powerful SDFS class to access SD
cards, this SD.h wrapper allows for use of existing Arduino libraries
which are built to only with with that SD class.
Additional helper functions added to ESP8266 native Filesystem:: classes
to help support this portability.
The rewrite is good enough to run the original SDWebServer and SD
example code without any changes.
* Add a FSConfig and SDFSConfig param to FS.begin()
Allows for configuration values to be passed into a filesystem via the
begin method. By default, a FS will receive a nullptr and should so
whatever is appropriate.
The base FSConfig class has one parameter, _autoFormat, set by the
default constructor to true.
For SPIFFS, you can now disable auto formatting on mount failure by
passing in a FSConfig(false) object.
For SDFS a SDFSConfig parameter can be passed into config specifying the
chip select and SPI configuration. If nothing is passed in, the begin
will fail since there are no safe default values here.
* Add FS::setConfig to set FS-specific options
Add a new call, FS::setConfig(const {SDFS,SPIFFS}Config *cfg), which
takes a FS-specific configuration object and copies any special settings
on a per-FS basis. The call is only valid on unmounted filesystems, and
checks the type of object passed in matches the FS being configured.
Updates the docs and tests to utilize this new configuration method.
* Add ::truncate to File interface
Fixes#3846
* Use polledTimeout for formatting yields, cleanup
Use the new polledTimeout class to ensure a yield every 5ms while
formatting.
Add in default case handling and some debug messages when invalid inputs
specified.
* Make setConfig take const& ref, cleaner code
setConfig now can take a parameter defined directly in the call by using
a const &ref to it, leading to one less line of code to write and
cleaner reading of the code.
Also clean up SDFS implementation pointer definition.
This speeds up the ESP8266WebServer::streamFile more than 3 times. Tested on streaming the 800+ Kb file from SD (FAT32), average time without a fix was 9000 ms, with the fix is 2600 ms (maximal possible SPI speed used), which is as fast as streaming the same file from internal SPIFFS. Hardware: WeMos D1 mini.
Use platform.local.txt to add -Werror to GCC for the build of all
code. Any warnings on a submitted patch will cause an error.
Several examples and libraries had warnings/errors (missing returns
on functions, types, etc.). Clean those up with this commit as well.
Use unique name values in library.properties for the esp8266 libraries
that are also Arduino IDE Built-In libraries. This solves the issue of
SD and Servo libraries always appearing as Type: Updatable in the
Arduino IDE 1.6.6 Library Manager when an esp8266 board is selected.
* master: (414 commits)
Don't export sketch if the underlying core does not support it. Fixes#3171
RSyntaxTextArea: using a modified version, tracked at https://github.com/arduino/RSyntaxTextArea. Fixes#3099
Updated keywords.txt
New editor on MacOSX: since CMD+J is known as "jump to selection" and the editor has no such feature, CMD+J is disabled on mac. See #3098
Old Preferences class remains for backwards compatibility as a delegate for PreferencesData
New Preferences window: renders fine on every OS and it's easier to adapt using NetBeans as visual editor. Fixes#3140
Remove spawn from exec command
Removed redundant call to File.deleteIfExists()
Removed buggy redundant check in FileUtils.deleteIfExists()
Restored current line/current selected lines display on lower left of the IDE. Fixes#3134
Updated cursor.ino
New editor on MacOSX: restored CMD+E for finding selected text
New editor on MacOSX: CMD+UP/DOWN moves cursor to start or end of sketch. See #3098
New editor on MacOSX: CMD+BACKSPACE deletes current line until cursor position, ALT+BACKSPACE deletes previous word. See #3098
ArduinoIDE is in the default package. Removed
Fixes #2969:
Fix Uncategorized warning message
New editor: ALT+ BACKSPACE deletes next word (OSX only). See #3098
New editor: ALT+ UP/DOWN move current line only if "editor.advanced" (hidden pref) is true. Fixes#3101
New editor: mark occurrences enable when "editor.advanced" (hidden pref) is true. Fixes#3102
...
Conflicts:
.gitignore
build/build.xml
hardware/esp8266com/esp8266/libraries/ESP8266WiFi/keywords.txt
hardware/esp8266com/esp8266/libraries/ESP8266WiFi/library.properties
hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/ESP8266WiFi.h
libraries/WiFi/README.adoc
libraries/WiFi/src/WiFi.cpp
libraries/WiFi/src/WiFiClient.cpp
libraries/WiFi/src/WiFiClient.h
libraries/WiFi/src/WiFiServer.cpp
libraries/WiFi/src/WiFiUdp.cpp
* commit 'b8b2869753d488bfd203615637e1de3912589a92': (68 commits)
Updated revisions.txt
Fixed wrong path in successful message
arduino-core: restored debug info in class files
Fixed NPE in case of missing boardData
Added missing translations
Added windows drivers
Added warning for uncertified boards
Update revision.txt
update revisions.txt
SoftwareSerial: match bool API with HardwareSerial
Fix to save as to parent folder is needed by all OSs, not just Mac
Update revision.txt
MacOSX: previous better IDE was missing some pieces. Added
MacOSX: better IDE
Updated revisions.txt
NEW button now behaves as clicking File -> New menu entry. Fixes#2685
Windows: bundled JRE updated to 8u31 build.xml now uses unzip target
Added .getParentFile() to saveas for mac. This prevents saving into the sketch itself
Compound edits weren't part of the undo/redo dance
An undoable action marks the sketh as modified
...
* master: (148 commits)
Update revision log
Cherry picked fix from 87865ac19df2f55e3d073cf5dc7ba08c0de4c584
Updated revision log
Added dependencies for AStylej.dll
Updated translations
Update revision log
Temporary fix for pulseIn() regression.
Added README.adoc for the library manager project
Fixed some libraries metadata.
Temporary disabled DefaultTargetTest under certain conditions
Updated translation from transifex
Updated some translation strings
Fixed test
sam: Fixed initialization of UART/USART mode register
update revision log
Fixed NPE when import menu are empty
Fixed NPE when currently selected platform is no more installed.
Optimized FileUtils.recursiveDelete(File) function
Fixed a bunch of simple warnings in java code
Removed unused classes Commander.java and Webserver.java
...
This includes empty stubs for most core Arduino functions.
Need to actually implement all those digital reads writes whatever.
Need to prebuild toolchains (xtensa-elf-lx106) for 3 platforms and put them on some download server.
Need to do the same with esptool.
Need to fix 0x40000 binary generation and add correct upload commands.
Maybe even implement uploads over WiFi.
