mirror of
https://github.com/esp8266/Arduino.git
synced 2025-04-25 20:02:37 +03:00
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Earle F. Philhower, III
|
1d0bc5efdf
|
Allow GZIP compressed flash updates (#6820)
* Allow GZIP compressed flash updates Modified the bootloader to be able to take stored updates in compressed GZIP format (i.e. the output of "gzip -9 xxx.bin") and decompress them on-the-fly to their final destination. This can work for apps and for filesystems (when used with the 2-step update option). Allow eboot to be built using -Os/2 optimizations by fixing some portions which failed when any optimizations were used. Add -Wall and use data and function sections to reduce size. Use -Os to minimize size. Remove obsolete esptool-ck calls to build a .ROM image, we don't use it. Move all uninitted variables to RAM from IRAM, allowing 8-bit access. Hook in @d-a-v and @pfalcon's uzlib port to actually do the decompression. Do not use any CRC checking which saves space. Since we have overwritten all of flash by the time we know id the CRC matches, there's nothing we could have done anyway. Adjust the Updater class to support GZIP files and not attempt to patch them. Bootloader builds to 0xd90 out of 0xfff bytes. * Add @d-a-v's patch for httpupdate https://github.com/esp8266/Arduino/pull/6820#pullrequestreview-326541014 * Update uzlib to point to pfalcon++ For now, because there are some self-test failures with @d-a-v's esp8266 branch (whose cool new features we don't actually use in eboot now) start with pfalcon's 2.9 release and add the 2 patches (clcidx to code from IRAM/RODATA, and the Windows test file renaming) needed to build and run successfully. * Add (c) notice for uzlib to README |
||
|
Earle F. Philhower, III
|
a389a995fb |
Add LittleFS as an optional filesystem, API compatible w/SPIFFS (but not on-flash-format compatible) (#5511)
* Add LittleFS as internal flash filesystem Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem, https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard flash utilizing the exact same API as the existing SPIFFS filesystem. LittleFS is built for low memory systems that are subject to random power losses, is actively supported by the ARMmbed community, supports directories, and seems to be much faster in the large-ish read-mostly applications I use. LittleFS, however, has a larger minimum file allocation unit and does not do static wear levelling. This means that for systems that need many little files (<4K), have small SPIFFS areas (64K), or which have a large static set of files covering the majority of flash coupled with a frequently updated set of other files, it may not perform as well. Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch, use LittleFS.open in place of SPIFFS.open to open files, and everything else just works thanks to the magic of @igrr's File base class. **LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS** Since it is a completely different filesystem, you will need to reformat your flash (and lose any data therein) to use it. Tools to build the flash filesystem and upload are at https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and https://github.com/earlephilhower/mklittlefs/ . The mklittlefs tool is installed as part of the Arduino platform installation, automatically. The included example shows a contrived read-mostly example and demonstrates how the same calls work on either SPIFFS.* or LittleFS.* Host tests are also included as part of CI. Directories are fully supported in LittleFS. This means that LittleFS will have a slight difference vs. SPIFFS when you use LittleFS.openDir()/Dir.next(). On SPIFFS dir.next() will return all filesystem entries, including ones in "subdirs" (because in SPIFFS there are no subdirs and "/" is the same as any other character in a filename). On LittleFS, dir.next() will only return entries in the directory specified, not subdirs. So to list files in "/subdir/..." you need to actually openDir("/subdir") and use Dir.next() to parse through just those elements. The returned filenames also only have the filename returned, not full paths. So on a FS with "/a/1", "/a/2" when you do openDir("/a"); dir.next().getName(); you get "1" and "2" and not "/a/1" and "/a/2" like in SPIFFS. This is consistent with POSIX ideas about reading directories and more natural for a FS. Most code will not be affected by this, but if you depend on openDir/Dir.next() you need to be aware of it. Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile, ::openNextFile, and ::rewind methods added to Filesystem objects. Documentation has been updated with this and other LittleFS information. Subdirectories are made silently when they do not exist when you try and create a file in a subdir. They are silently removed when the last file in them is deleted. This is consistent with what SPIFFS does but is obviously not normal POSIX behavior. Since there has never been a "FS.mkdir()" method this is the only way to be compatible with legacy SPIFFS code. SPIFFS code has been refactored to pull out common flash_hal_* ops and placed in its own namespace, like LittleFS. * Fix up merge blank line issue * Merge in the FSConfig changs from SDFS PR Enable setConfig for LittleFS as well plys merge the SPIFFS changes done in the SDFS PR. * Fix merge errors * Update to use v2-alpha branch The V2-alpha branch supports small file optimizations which can help increase the utilization of flash when small files are prevalent. It also adds support for metadata, which means we can start adding things like file creation times, if desired (not yet). * V2 of littlefs is now in upstream/master * Update test to support non-creation-ordered files In a directory, the order in which "readNextFile()" will return a name is undefined. SPIFFS may return it in order, but LittleFS does not as of V2. Update the test to look for files by name when doing readNextFile() testing. * Fix LittleFS.truncate implementation * Fix SDFS tests SDFS, SPIFFS, and LittleFS now all share the same common set of tests, greatly increasing the SDFS test coverage. * Update to point to mklittlefs v2 Upgrade mklittlefs to V2 format support * Remove extra FS::write(const char *s) method This was removed in #5861 and erroneously re-introduced here. * Minimize spurious differences from master * Dramatically reduce memory usage Reduce the program and read chunk sizes which impacts performance minimally but reduces per-file RAM usage of 16KB to <1KB. * Add @d-a-v's host emulation for LittleFS * Fix SW Serial library version * Fix free space reporting Thanks to @TD-er for discovering the issue * Update littlefs to latest upstream * Remove sdfat version included by accident * Update SDFAT to include MOCK changes required * Update to include SD.h test of file append |
||
|
Earle F. Philhower, III
|
b1da9eda46
|
SD Filesystem compatible with 8266 File, using latest SdFat (#5525)
* Add a FAT filesystem for SD cards to Arduino FS
Arduino forked a copy of SD lib several years ago, put their own wrapper
around it, and it's been languishing in our ESP8266 libraries ever since
as SD. It doesn't support long file names, has class names which
conflict with the ESP8266 internal names, and hasn't been updated in
ages.
The original author of the SD library has continued work in the
meantime, and produced a very feature rich implementation of SdFat. It
unfortunately also conflicts with the class names we use in ESP8266
Arduino and has a different API than the internal SPIFFS or proposed
LittleFS filesystem objects.
This PR puts a wrapper around the latest and greatest SdFat library,
by forking it and wrapping its classes in a private namespace "sdfat,"
and making as thin a wrapper as possible around it to conform to
the ESP8266 FS, File, and Dir classes.
This PR also removes the Arduino SD.h class library and rewrites it
using the new SDFS filesystem to make everything in the ESP8266
Arduino core compatible with each other.
By doing so it lets us use a single interface for anything needing a
file instead of multiple ones (see SDWebServer and how a different
object is needed vs. one serving from SPIFFS even though the logic is
all the same). Same for BearSSL's CertStores and probably a few others
I've missed, cleaning up our code base significantly.
Like LittleFS, silently create directories when a file is created with
a subdirectory specifier ("/path/to/file.txt") if they do not yet exist.
Adds a blacklist of sketches to skip in the CI process (because SdFat
has many examples which do not build properly on the ESP8266).
Now that LittleFS and SDFS have directory support, the FS needs to be
able to communicate whether a name is one or the other. Add a simple
bool FS::isDirectory() and bool FS::isFile() method. SPIFFS doesn't
have directories, so if it's valid it's a file and reported as such.
Add ::mkdir/::rmdir to the FS class to allow users to make and destroy
subdirectories. SPIFFS directory operations will, of course, fail
and return false.
Emulate a 16MB SD card and allow test runner to exercise it by using
a custom SdFat HOST_MOCK-enabled object.
Throw out the original Arduino SD.h class and rewrite from scratch using
only the ESP8266 native SDFS calls. This makes "SD" based applications
compatible with normal ESP8266 "File" and "FS" and "SPIFFS" operations.
The only major visible change for users is that long filenames now are
fully supported and work without any code changes. If there are static
arrays of 11 bytes for old 8.3 names in code, they will need to be
adjusted.
While it is recommended to use the more powerful SDFS class to access SD
cards, this SD.h wrapper allows for use of existing Arduino libraries
which are built to only with with that SD class.
Additional helper functions added to ESP8266 native Filesystem:: classes
to help support this portability.
The rewrite is good enough to run the original SDWebServer and SD
example code without any changes.
* Add a FSConfig and SDFSConfig param to FS.begin()
Allows for configuration values to be passed into a filesystem via the
begin method. By default, a FS will receive a nullptr and should so
whatever is appropriate.
The base FSConfig class has one parameter, _autoFormat, set by the
default constructor to true.
For SPIFFS, you can now disable auto formatting on mount failure by
passing in a FSConfig(false) object.
For SDFS a SDFSConfig parameter can be passed into config specifying the
chip select and SPI configuration. If nothing is passed in, the begin
will fail since there are no safe default values here.
* Add FS::setConfig to set FS-specific options
Add a new call, FS::setConfig(const {SDFS,SPIFFS}Config *cfg), which
takes a FS-specific configuration object and copies any special settings
on a per-FS basis. The call is only valid on unmounted filesystems, and
checks the type of object passed in matches the FS being configured.
Updates the docs and tests to utilize this new configuration method.
* Add ::truncate to File interface
Fixes #3846
* Use polledTimeout for formatting yields, cleanup
Use the new polledTimeout class to ensure a yield every 5ms while
formatting.
Add in default case handling and some debug messages when invalid inputs
specified.
* Make setConfig take const& ref, cleaner code
setConfig now can take a parameter defined directly in the call by using
a const &ref to it, leading to one less line of code to write and
cleaner reading of the code.
Also clean up SDFS implementation pointer definition.
|
||
|
Earle F. Philhower, III
|
ead83c16a6 |
Add .git to BearSSL repo reference (#5827)
The BearSSL submodule reference is missing the .git extension on the URL. Add it, to match the rest of the repos. |
||
|
Earle F. Philhower, III
|
9790e1cb7c |
Use esptool.py to handle sketch upload, make python available on Windows, too (#5635)
* Add esptool.py, pyserial, and python to JSON Add installation of python on Win32/Win64, and on all systems install esptool.py and pyserial. * Initial esptool.py upload test * First successfull esptool.py upload * Patch in verbose flag operation * Replace esptool-ck.exe with Python equivalent Remove need for binary esptool-ck.exe by implementing the same logic as esptool-ck uses in Python. Only image creation is supported, and only in the Arduino standard mode (with its custom bootloader and ROM layout). * Remove all esptool-ck.exe, hook Windows Python Remove all references to esptool-ck and use Python on Windows and Linux for all recipes where possible. * Use python to make core_version as well Avoid ugly bash and CMD.exe tricks in platform.txt by using python to make the core_version header. * Rename conflicting script, clean up packager * Windows test passes Need to make sure Python2 and Python3 compatible and paths are munged properly to avoid eaccidentally escaping things when calling esptool.py Able to compile, build a BIN and upload via esptool.py on a Windows machine without Python installed globally, only as part of the Arduino tools package. * Use github sources for pyserial * Erase calibration or all flash before programming Add back in erase support by calling esptool.py twice (since it does not support chained operations like esptool-ck.exe). * Make 460K default speed, remove 961K 961K doesn't seem to work with esptool, so make 460K the default upload speed and remove 961K. Even at this lower speed, esptool.py is much faster to upload (even before taking into account the compression when doing things like SPIFFS and code upload). * Make erase and upload work again Arduino does not support a upload.#.cmd pattern, so we need to do everything in a single command line. Make it cleaner by introducing a Python wrapper script which will run the same executable with different sets of commands (since we need to erase a block w/a separate invocation from the real upload). Update boards.txt to use the new options format, placing the esptool command as "version" when there is no "erase_flash" or "erase_region" to be done to keep things simple. * Move esptool/pyserial to submodules Since esptool.py and pyserial are coming directly from github repos, there is no need to include them as a tool in package.json. * Restore 921K upload opt, silent downgrade to 460k To enable full backward compatibility, restore the 921k option for upload speed but silently change it to 460k in the upload.py script. Add error checking on upload.py |
||
david gauchard
|
4c8d8f1e8a |
uart: BW improvements (#4620)
* uart fixes and BW improvements * uart: read_char straightly use hw buffer * +attributes for functions called by ISR * uart: BW improvements read_char straightly use hw buffer (+ ~10%bw) read by block (+ ~190%bw) (instead of generic Stream::readBytes) attributes for functions called by ISR remove overrun message remove some ISR flags which were not honoured * fix merge * fix buffer overflow * serial stress test sketch * astyle * serial stress example: interactive keyboard, stop reading, overrun * serial device test: bandwidth & overrun * update + HardwareSerial::hasError() * interactive overrun in example * astyle * Test using @plerup's SoftwareSerial as submodule (tag 3.4.1) * update upstream ref (fix warning) * host mock uart/read(buf,size) * reset style changes in submodules before style diff * update build_boards_manager_package.sh for submodules * trigger CI (removing space) * cannot reproduce locally the CI issue, setting bash -x option to get live trace * remove previously added (in this PR) 'set -e' in package builder (passes local tests, not real CI) script-comment new recipe.hooks.core.prebuild.3 (along with already commented .1 and .2) moved CI package test to be first on the test list remove 'set -x', wish me luck |
||
|
Earle F. Philhower, III
|
5a033835e1
|
Add a build directory for libbearssl.a (#4736)
Simple git submodule and makefile for building the bearssl library from source in the Arduino tree. |
||
|
david gauchard
|
32f6826dcf | lwip2 integration |