mirror of
https://github.com/esp8266/Arduino.git
synced 2025-04-19 23:22:16 +03:00
a389a995fb
* Add LittleFS as internal flash filesystem Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem, https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard flash utilizing the exact same API as the existing SPIFFS filesystem. LittleFS is built for low memory systems that are subject to random power losses, is actively supported by the ARMmbed community, supports directories, and seems to be much faster in the large-ish read-mostly applications I use. LittleFS, however, has a larger minimum file allocation unit and does not do static wear levelling. This means that for systems that need many little files (<4K), have small SPIFFS areas (64K), or which have a large static set of files covering the majority of flash coupled with a frequently updated set of other files, it may not perform as well. Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch, use LittleFS.open in place of SPIFFS.open to open files, and everything else just works thanks to the magic of @igrr's File base class. **LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS** Since it is a completely different filesystem, you will need to reformat your flash (and lose any data therein) to use it. Tools to build the flash filesystem and upload are at https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and https://github.com/earlephilhower/mklittlefs/ . The mklittlefs tool is installed as part of the Arduino platform installation, automatically. The included example shows a contrived read-mostly example and demonstrates how the same calls work on either SPIFFS.* or LittleFS.* Host tests are also included as part of CI. Directories are fully supported in LittleFS. This means that LittleFS will have a slight difference vs. SPIFFS when you use LittleFS.openDir()/Dir.next(). On SPIFFS dir.next() will return all filesystem entries, including ones in "subdirs" (because in SPIFFS there are no subdirs and "/" is the same as any other character in a filename). On LittleFS, dir.next() will only return entries in the directory specified, not subdirs. So to list files in "/subdir/..." you need to actually openDir("/subdir") and use Dir.next() to parse through just those elements. The returned filenames also only have the filename returned, not full paths. So on a FS with "/a/1", "/a/2" when you do openDir("/a"); dir.next().getName(); you get "1" and "2" and not "/a/1" and "/a/2" like in SPIFFS. This is consistent with POSIX ideas about reading directories and more natural for a FS. Most code will not be affected by this, but if you depend on openDir/Dir.next() you need to be aware of it. Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile, ::openNextFile, and ::rewind methods added to Filesystem objects. Documentation has been updated with this and other LittleFS information. Subdirectories are made silently when they do not exist when you try and create a file in a subdir. They are silently removed when the last file in them is deleted. This is consistent with what SPIFFS does but is obviously not normal POSIX behavior. Since there has never been a "FS.mkdir()" method this is the only way to be compatible with legacy SPIFFS code. SPIFFS code has been refactored to pull out common flash_hal_* ops and placed in its own namespace, like LittleFS. * Fix up merge blank line issue * Merge in the FSConfig changs from SDFS PR Enable setConfig for LittleFS as well plys merge the SPIFFS changes done in the SDFS PR. * Fix merge errors * Update to use v2-alpha branch The V2-alpha branch supports small file optimizations which can help increase the utilization of flash when small files are prevalent. It also adds support for metadata, which means we can start adding things like file creation times, if desired (not yet). * V2 of littlefs is now in upstream/master * Update test to support non-creation-ordered files In a directory, the order in which "readNextFile()" will return a name is undefined. SPIFFS may return it in order, but LittleFS does not as of V2. Update the test to look for files by name when doing readNextFile() testing. * Fix LittleFS.truncate implementation * Fix SDFS tests SDFS, SPIFFS, and LittleFS now all share the same common set of tests, greatly increasing the SDFS test coverage. * Update to point to mklittlefs v2 Upgrade mklittlefs to V2 format support * Remove extra FS::write(const char *s) method This was removed in #5861 and erroneously re-introduced here. * Minimize spurious differences from master * Dramatically reduce memory usage Reduce the program and read chunk sizes which impacts performance minimally but reduces per-file RAM usage of 16KB to <1KB. * Add @d-a-v's host emulation for LittleFS * Fix SW Serial library version * Fix free space reporting Thanks to @TD-er for discovering the issue * Update littlefs to latest upstream * Remove sdfat version included by accident * Update SDFAT to include MOCK changes required * Update to include SD.h test of file append
SPIFFS (SPI Flash File System)
V0.3.7
Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976 at gmail.com)
For legal stuff, see LICENSE. Basically, you may do whatever you want with the source. Use, modify, sell, print it out, roll it and smoke it - as long as I won't be held responsible.
Love to hear feedback though!
INTRODUCTION
Spiffs is a file system intended for SPI NOR flash devices on embedded targets.
Spiffs is designed with following characteristics in mind:
- Small (embedded) targets, sparse RAM without heap
- Only big areas of data (blocks) can be erased
- An erase will reset all bits in block to ones
- Writing pulls one to zeroes
- Zeroes can only be pulled to ones by erase
- Wear leveling
BUILDING
mkdir build; make
Otherwise, configure the builddir variable towards the top of makefile as something opposed to the default build. Sanity check on the host via make test and refer to .travis.yml for the official in-depth testing procedure. See the wiki for integrating spiffs into projects and spiffsimg from nodemcu is a good example on the subject.
FEATURES
What spiffs does:
- Specifically designed for low ram usage
- Uses statically sized ram buffers, independent of number of files
- Posix-like api: open, close, read, write, seek, stat, etc
- It can run on any NOR flash, not only SPI flash - theoretically also on embedded flash of a microprocessor
- Multiple spiffs configurations can run on same target - and even on same SPI flash device
- Implements static wear leveling
- Built in file system consistency checks
- Highly configurable
What spiffs does not:
- Presently, spiffs does not support directories. It produces a flat structure. Creating a file with path tmp/myfile.txt will create a file called tmp/myfile.txt instead of a myfile.txt under directory tmp.
- It is not a realtime stack. One write operation might last much longer than another.
- Poor scalability. Spiffs is intended for small memory devices - the normal sizes for SPI flashes. Going beyond ~128Mbyte is probably a bad idea. This is a side effect of the design goal to use as little ram as possible.
- Presently, it does not detect or handle bad blocks.
- One configuration, one binary. There's no generic spiffs binary that handles all types of configurations.
MORE INFO
See the wiki for configuring, integrating, using, and optimizing spiffs.
For design, see docs/TECH_SPEC.
For a generic spi flash driver, see this.
HISTORY
0.3.7
- fixed prevent seeking to negative offsets #158
- fixed file descriptor offsets not updated for multiple fds on same file #157
- fixed cache page not closed for removed files #156
- fixed a lseek bug when seeking exactly to end of a fully indexed first level LUT #148
- fixed wear leveling issue #145
- fixed attempt to write out of bounds in flash #130,
- set file offset when seeking over end #121 (thanks @sensslen)
- fixed seeking in virgin files #120 (thanks @sensslen)
- Optional file metadata #128 (thanks @cesanta)
- AFL testing framework #100 #143 (thanks @pjsg)
- Testframe updates
New API functions:
SPIFFS_update_meta, SPIFFS_fupdate_meta- updates metadata for a file
New config defines:
SPIFFS_OBJ_META_LEN- enable possibility to add extra metadata to files
0.3.6
- Fix range bug in index memory mapping #98
- Add index memory mapping #97
- Optimize SPIFFS_read for large files #96
- Add temporal cache for opening files #95
- More robust gc #93 (thanks @dismirlian)
- Fixed a double write of same data in certain cache situations
- Fixed an open bug in READ_ONLY builds
- File not visible in SPIFFS_readdir #90 (thanks @benpicco-tmp)
- Cache load code cleanup #92 (thanks @niclash)
- Fixed lock/unlock asymmetry #88 #87 (thanks @JackJefferson, @dpruessner)
- Testframe updates
New API functions:
SPIFFS_ix_map- map index meta data to memory for a fileSPIFFS_ix_unmap- unmaps index meta data for a fileSPIFFS_ix_remap- changes file offset for index metadata mapSPIFFS_bytes_to_ix_map_entries- utility, get length of needed vector for given amount of bytesSPIFFS_ix_map_entries_to_bytes- utility, get number of bytes a vector can represent given length
New config defines:
SPIFFS_IX_MAP- enable possibility to map index meta data to memory for reading fasterSPIFFS_TEMPORAL_FD_CACHE- enable temporal cache for opening files fasterSPIFFS_TEMPORAL_CACHE_HIT_SCORE- for tuning the temporal cache
0.3.5
- Fixed a bug in fs check
- API returns actual error codes #84) (thanks @Nails)
- Fix compiler warnings for non-gcc #83 #81 (thanks @Nails)
- Unable to recover from full fs #82 (thanks @rojer)
- Define SPIFFS_O_* flags #80
- Problem with long filenames #79 (thanks @psjg)
- Duplicate file name bug fix #74 (thanks @igrr)
- SPIFFS_eof and SPIFFS_tell return wrong value #72 (thanks @ArtemPisarenko)
- Bunch of testframe updates #77 #78 #86 (thanks @dpreussner, @psjg a.o)
0.3.4
- Added user callback file func.
- Fixed a stat bug with obj id.
- SPIFFS_probe_fs added
- Add possibility to compile a read-only version of spiffs
- Make magic dependent on fs length, if needed (see #59 & #66) (thanks @hreintke)
- Exposed SPIFFS_open_by_page_function
- Zero-size file cannot be seek #57 (thanks @lishen2)
- Add tell and eof functions #54 (thanks @raburton)
- Make api string params const #53 (thanks @raburton)
- Preserve user_data during mount() #51 (thanks @rojer)
New API functions:
SPIFFS_set_file_callback_func- register a callback informing about file eventsSPIFFS_probe_fs- probe a spi flash trying to figure out size of fsSPIFFS_open_by_page- open a file by page indexSPIFFS_eof- checks if end of file is reachedSPIFFS_tell- returns current file offset
New config defines:
SPIFFS_READ_ONLYSPIFFS_USE_MAGIC_LENGTH
0.3.3
Might not be compatible with 0.3.2 structures. See issue #40
- Possibility to add integer offset to file handles
- Truncate function presumes too few free pages #49
- Bug in truncate function #48 (thanks @PawelDefee)
- Update spiffs_gc.c - remove unnecessary parameter (thanks @PawelDefee)
- Update INTEGRATION docs (thanks @PawelDefee)
- Fix pointer truncation in 64-bit platforms (thanks @igrr)
- Zero-sized files cannot be read #44 (thanks @rojer)
- (More) correct calculation of max_id in obj_lu_find #42 #41 (thanks @lishen2)
- Check correct error code in obj_lu_find_free #41 (thanks @lishen2)
- Moar comments for SPIFFS_lseek (thanks @igrr)
- Fixed padding in spiffs_page_object_ix #40 (thanks @jmattsson @lishen2)
- Fixed gc_quick test (thanks @jmattsson)
- Add SPIFFS_EXCL flag #36
- SPIFFS_close may fail silently if cache is enabled #37
- User data in callbacks #34
- Ignoring SINGLETON build in cache setup (thanks Luca)
- Compilation error fixed #32 (thanks @chotasanjiv)
- Align cand_scores (thanks @hefloryd)
- Fix build warnings when SPIFFS_CACHE is 0 (thanks @ajaybhargav)
New config defines:
SPIFFS_FILEHDL_OFFSET
0.3.2
- Limit cache size if too much cache is given (thanks pgeiem)
- New feature - Controlled erase. #23
- SPIFFS_rename leaks file descriptors #28 (thanks benpicco)
- moved dbg print defines in test framework to params_test.h
- lseek should return the resulting offset (thanks hefloryd)
- fixed type on dbg ifdefs
- silence warning about signed/unsigned comparison when spiffs_obj_id is 32 bit (thanks benpicco)
- Possible error in test_spiffs.c #21 (thanks yihcdaso-yeskela)
- Cache might writethrough too often #16
- even moar testrunner updates
- Test framework update and some added tests
- Some thoughts for next gen
- Test sigsevs when having too many sectors #13 (thanks alonewolfx2)
- GC might be suboptimal #11
- Fix eternal readdir when objheader at last block, last entry
New API functions:
SPIFFS_gc_quick- call a nonintrusive gcSPIFFS_gc- call a full-scale intrusive gc
0.3.1
- Removed two return warnings, was too triggerhappy on release
0.3.0
- Added existing namecheck when creating files
- Lots of static analysis bugs #6
- Added rename func
- Fix SPIFFS_read length when reading beyond file size
- Added reading beyond file length testcase
- Made build a bit more configurable
- Changed name in spiffs from "errno" to "err_code" due to conflicts compiling in mingw
- Improved GC checks, fixed an append bug, more robust truncate for very special case
- GC checks preempts GC, truncate even less picky
- Struct alignment needed for some targets, define in spiffs config #10
- Spiffs filesystem magic, definable in config
New config defines:
SPIFFS_USE_MAGIC- enable or disable magic check upon mountSPIFFS_ALIGNED_OBJECT_INDEX_TABLES- alignment for certain targets
New API functions:
SPIFFS_rename- rename filesSPIFFS_clearerr- clears last errnoSPIFFS_info- returns info on used and total bytes in fsSPIFFS_format- formats the filesystemSPIFFS_mounted- checks if filesystem is mounted