* Add fileCreation/getCreation create-time accessors
For SDFS and LittleFS, enable a creation time accessor for files and Dir
iterators, similar to the existing fileTime/getLastWrite calls.
Remove spurious Dir::getLastWrite method (the proper and only documented
way is really Dir::fileTime).
Update json to point to new mklittlefs which copies the creation date of
files to the image.
Fixes#6992
* Remove malloc(), use stack vars for temp names
LFS filenames are limited in size and generally very small. Use a stack
variable instead of a dynamic allocation when performing full-path
computations.
* Replace "Creation" w/"CreationTime" in FS accessor
Per review, `getCreation` -> `getCreationTime`, `fileCreation` ->
`fileCreationTime`.
The names `fileTime()` and `getLastWrite()` are inherited from ESP32
implementation and unchanged.
* Add creation time to listfiles SD example
* Enable SdFat's sateTime callback for timestamping
SdFat requries the dateTimeCallback call (global for everything) to
update dates and times on created files.
Because the callback signature doesn't have space for us to provide
any parameters, we cannot get the the File, Dir, or FS object's
dateTimeCB member. Instead, just go with `time(null)` as the callback
function which is right in all but the most esoteric cases.
* Correct DOS year/month offset in dateTime callback
* Fix docs to match new xxxCreationTime() API names
Co-authored-by: Develo <deveyes@gmail.com>
* Use a python3 script to call python3
It's odd, but because Windows requires a full Python3 install we must
have an executable called "tools/python3/python3" to use Python3 in the
toolchain.
Before, we simply symlinked to /usr/bin/python3 (for Linux) or
/usr/local/bin/python3 (Mac). Unfortunately, depending on the method of
installation, on MacOS the Python3 executable can be in /usr/bin/python3
instead.
To avoid the entire issue, unify the Mac and Linux python3 placeholders
to use python3 itself to jump to the real executable.
Fixes#6931
* Explicitly remove old symlink to python3
The tar extraction for the updated python3 tarball will fail on systems
that already have a symlink in /tools/python3/python3 because the tar
extractor attempts to open the *target of the symlink* (i.e. the actual
interpreter in /usr/bin or /usr/local/bin).
Add a commented hack to destroy this symlink before expanding the
tarballs, if the file exists. This is safe to do since it will be
overwritten by any extractions of the python3 tarball later in the
process.
Co-authored-by: david gauchard <gauchard@laas.fr>
Fixes#7006
mklittlefs for 32-bit Linux disappeared from the packages.json. Looks
like a transient build problem on the older eqt release (later builds
look fine). Add it back, pointing to the first successful mklittlefs
build for lin32.
Co-authored-by: Develo <deveyes@gmail.com>
* adds initial commit to boards.txt before actual edits
* reminder to remove vim .swp file
* Removes spaces and modifies outdated tags for menu `CpuFrequency -> xtal` for example
* Adds SparkFun Bynk Board to boards.txt.py, moves .build flags all together for SparkFun Blynk
* runs boards.txt.py to generate files for pull request - all files have been included as instructed at top of boards.txt file
* Deletes three .orig files generated by boards.txt.py: rst, txt, and json
* Moves boards.txt.py back to /tools directory and attempts to change its mode.
* restore 'x' flag
* Add time to filesystem API
Support the ESP32 File::getLastWrite() call and setting the time on
all filesystems automatically (assuming the system clock has
been set properly and time(NULL) returns the proper time!).
Adds Dir::fileTime() to get the time of a file being listed, similar to
Dir::fileName() and Dir::fileSize().
Adds ::setTimeCallback(time_t (*cb)()) to File, Dir, and FS, allowing
users to override the default timestamp on a per-file, directory, or
filesystem basis. By default, a simple callback returning time(nullptr)
is implemented.
LittleFS uses the 't' attribute and should be backwards compatible.
SD/SDFS work and include wrappers for obsolete SdFat timestamp callbacks
using the MSDOS time.
This PR does not update SPIFFS, due to compatability concerns and a
possible massive rewrite which would make it possible to determine if an
old-style ot metadata enabled FS is present at mount time.
Includes an updated SD/listfiles and LittleFS_time example.
Replaces #6315
* Add links to new mklittlefs w/timestamp support
Include the update mklittlefs which generated 't' metadata on imported
files.
../tools/sdk/lwip2/include/netif/lowpan6_opts.h
* Add explicit note about timestamp being local time
* Address review concerns
Clean up some awkward object instantiations.
Remove the _enableTime flag/setter from SPIFFS.
Clean up the FSConfig constructors using C++ style init lists.
When the FS_END was adjusted to end on a full block (i.e. rounded down)
to avoid filesystem issues, but _FS_end was changed. The EEPROM library
used _FS_end to implicitly calculate the start of the EEPROM data, so
this means after the _FS_end fix, EEPROM data written with prior
releases would "disappear."
Avoid the issue by explicitly calculating the EEPROM start location in
the linker, using the same formula as prior release.
Fixes#6531
Build a single sketch using a Travis-CI OSX instance to validate the
toolchain works properly on Macs.
Update the installed python3 symlink to point to the proper spot for OSX
(Python3 is in /usr/local/bin, not /usr/bin).
Fixes#6490
The get.py renamer uses some logic which results in the Win32 extracted
directory always ending up as `tools/python` and not `tools/python3`.
Adjust the zip archive and title to work around this (and the IDE
proper) issue.
* Move all scripts and documentation to Python3
Python 2 EOL is Jan 1, 2020. Migrate scripts to run under Python 3.
Under Windows, we're already running Python 3.7, by dumb luck. The
oddness is that the Windows standalone executable for Python 3 is called
"python" whereas under UNIX-like OSes it's called "python3" with
"python" always referring to the Python 2 executable. The ZIP needs to
be updated to include a Python3.exe (copy of Python.exe) so that we can
use the same command lines under Linux and Windows, and to preserve my
sanity.
Fixes#6376
* Add new Windows ZIP with python3.exe file
* Sort options in boards.txt generation for repeatability
The order of the board opts dict changes depending on the Python version
and machine, so sort the options before printing them to get a stable
ordering.
* Re-add Python2 compatibility tweaks
Most scripts can run as Python 2 or Python 3 with minimal changes, so
re-add (and fix, as necessary) compatibility tweaks to the scripts.
* Upgrade to 2.5.0-4 toolchain w/improved pgm_read_x
Rebuild the entire toolchain (including standard libraries) with the
latest pgm_read_xxx headers included (which fix unaligned dword reads
from progmem and run faster/smaller, and a pgm_read_byte change which
removes an instruction on each read saving flash).
Pull in latest bearssl while we're at it, too, which speeds up EC
handshakes and reduced ROM usage, too.
* Fix C++ exceptions
Exception code now only does 32b aligned reads from progmem to access
the eh_table (some via -mforce-l32, some via hand-inserted pgm_read_x
macros).
Fixes#6151Fixes#6305Fixes#6198
Fixes#6068
Drop from the tools key all version:"1.20.0-26-gb404fb9" entries (which
were pre-2.0.0 and whose entry in platform versions was deleted on last
release).
* Prepare signing before sketch is compiled
This makes the right value available in ARDUINO_SIGNING in the sketch.
* Remove unnecessary .1s and fix packager script
Fixes#6220
MklittleFS had different configuration options which affected small
files and could result in crashes or corruption on upload.
Update mklittlefs tool to one that matches the config of the current
library.
You will need to re-run tools/get.py to get the new tool version if you
are running from Git.
* Add LittleFS as internal flash filesystem
Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem,
https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard
flash utilizing the exact same API as the existing SPIFFS filesystem.
LittleFS is built for low memory systems that are subject to random power
losses, is actively supported by the ARMmbed community, supports directories,
and seems to be much faster in the large-ish read-mostly applications I use.
LittleFS, however, has a larger minimum file allocation unit and does not do
static wear levelling. This means that for systems that need many little
files (<4K), have small SPIFFS areas (64K), or which have a large static
set of files covering the majority of flash coupled with a frequently
updated set of other files, it may not perform as well.
Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch,
use LittleFS.open in place of SPIFFS.open to open files, and everything
else just works thanks to the magic of @igrr's File base class.
**LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS**
Since it is a completely different filesystem, you will need to reformat
your flash (and lose any data therein) to use it. Tools to build the
flash filesystem and upload are at
https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and
https://github.com/earlephilhower/mklittlefs/ . The mklittlefs tool
is installed as part of the Arduino platform installation, automatically.
The included example shows a contrived read-mostly example and
demonstrates how the same calls work on either SPIFFS.* or LittleFS.*
Host tests are also included as part of CI.
Directories are fully supported in LittleFS. This means that LittleFS
will have a slight difference vs. SPIFFS when you use
LittleFS.openDir()/Dir.next(). On SPIFFS dir.next()
will return all filesystem entries, including ones in "subdirs"
(because in SPIFFS there are no subdirs and "/" is the same as any
other character in a filename).
On LittleFS, dir.next() will only return entries in the directory
specified, not subdirs. So to list files in "/subdir/..." you need
to actually openDir("/subdir") and use Dir.next() to parse through
just those elements. The returned filenames also only have the
filename returned, not full paths. So on a FS with "/a/1", "/a/2"
when you do openDir("/a"); dir.next().getName(); you get "1" and "2"
and not "/a/1" and "/a/2" like in SPIFFS. This is consistent with
POSIX ideas about reading directories and more natural for a FS.
Most code will not be affected by this, but if you depend on
openDir/Dir.next() you need to be aware of it.
Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile,
::openNextFile, and ::rewind methods added to Filesystem objects.
Documentation has been updated with this and other LittleFS information.
Subdirectories are made silently when they do not exist when you
try and create a file in a subdir. They are silently removed when
the last file in them is deleted. This is consistent with what
SPIFFS does but is obviously not normal POSIX behavior. Since there
has never been a "FS.mkdir()" method this is the only way to be
compatible with legacy SPIFFS code.
SPIFFS code has been refactored to pull out common flash_hal_* ops
and placed in its own namespace, like LittleFS.
* Fix up merge blank line issue
* Merge in the FSConfig changs from SDFS PR
Enable setConfig for LittleFS as well plys merge the SPIFFS changes
done in the SDFS PR.
* Fix merge errors
* Update to use v2-alpha branch
The V2-alpha branch supports small file optimizations which can help
increase the utilization of flash when small files are prevalent.
It also adds support for metadata, which means we can start adding
things like file creation times, if desired (not yet).
* V2 of littlefs is now in upstream/master
* Update test to support non-creation-ordered files
In a directory, the order in which "readNextFile()" will return a name
is undefined. SPIFFS may return it in order, but LittleFS does not as
of V2. Update the test to look for files by name when doing
readNextFile() testing.
* Fix LittleFS.truncate implementation
* Fix SDFS tests
SDFS, SPIFFS, and LittleFS now all share the same common set of tests,
greatly increasing the SDFS test coverage.
* Update to point to mklittlefs v2
Upgrade mklittlefs to V2 format support
* Remove extra FS::write(const char *s) method
This was removed in #5861 and erroneously re-introduced here.
* Minimize spurious differences from master
* Dramatically reduce memory usage
Reduce the program and read chunk sizes which impacts performance
minimally but reduces per-file RAM usage of 16KB to <1KB.
* Add @d-a-v's host emulation for LittleFS
* Fix SW Serial library version
* Fix free space reporting
Thanks to @TD-er for discovering the issue
* Update littlefs to latest upstream
* Remove sdfat version included by accident
* Update SDFAT to include MOCK changes required
* Update to include SD.h test of file append
To avoid issues as seen in 2.5.1 release where an invalid JSON was
published for a few hours, fail the board script if JQ can't parse the
generated output file.
* Upgrade to https: serving for JSON, links in docs
Fixes#5480
* Update boards.rst documentation
* Update more documentation http: refs to https:
* Remove obsolete staging info
* Drop obsolete versions from JSON programatically
After the final merge is done on the JSON, strip out any named versions
from the final product.
Removing 1.6.5-* and 2.5.0-beta(1,2,3) for now.
* Remove 2.4.0-rc(0/1) from JSON, too
* New menu option to minimize BSSL ROM with only RSA
Adds a menu option and define to limit BearSSL to older RSA connection
options. This saves ~45K program memory and can speed up connections
since EC, while more secure, is significantly slower on the chip.
The supported ciphers are identical to the ones that axTLS supported.
Fixes#6005
* Add default SSLFLAGS(blank) to platform.txt
* Fix unused variable warning
* Add clarifying comment to menu items
* Fix packaged python paths for Windows
Fixes#5881
The path to python.exe on Windows needs to change depending on whether
it is installed via the boards manager or GIT (similar to the compiler
paths). Adjust accordingly.
* Add python-placeholder to make boardsmanager happy
An empty "python" directory will be created by the boards-manager
installer. Required because all archs need all tools defined.
* Make the placeholder include a symlink for "python"
* Fix esptool windows python path
Fixes a typo in the python path for Windows for upload.
* Fix esptool windows python path
Fixes a typo in the python path for Windows for upload.
* Fix tools path when in GIT mode for upload
Simply import the pyserial and esptool modules directly into upload.py
instead of trying to fake things with os.fork()s. Reduces code and is
more Pythonic.
* Add esptool.py, pyserial, and python to JSON
Add installation of python on Win32/Win64, and on all systems install
esptool.py and pyserial.
* Initial esptool.py upload test
* First successfull esptool.py upload
* Patch in verbose flag operation
* Replace esptool-ck.exe with Python equivalent
Remove need for binary esptool-ck.exe by implementing the same logic as
esptool-ck uses in Python. Only image creation is supported, and only
in the Arduino standard mode (with its custom bootloader and ROM
layout).
* Remove all esptool-ck.exe, hook Windows Python
Remove all references to esptool-ck and use Python on Windows and Linux
for all recipes where possible.
* Use python to make core_version as well
Avoid ugly bash and CMD.exe tricks in platform.txt by using python to
make the core_version header.
* Rename conflicting script, clean up packager
* Windows test passes
Need to make sure Python2 and Python3 compatible and paths are munged
properly to avoid eaccidentally escaping things when calling esptool.py
Able to compile, build a BIN and upload via esptool.py on a Windows
machine without Python installed globally, only as part of the Arduino
tools package.
* Use github sources for pyserial
* Erase calibration or all flash before programming
Add back in erase support by calling esptool.py twice (since it does not
support chained operations like esptool-ck.exe).
* Make 460K default speed, remove 961K
961K doesn't seem to work with esptool, so make 460K the default upload
speed and remove 961K.
Even at this lower speed, esptool.py is much faster to upload (even
before taking into account the compression when doing things like SPIFFS
and code upload).
* Make erase and upload work again
Arduino does not support a upload.#.cmd pattern, so we need to do
everything in a single command line. Make it cleaner by introducing a
Python wrapper script which will run the same executable with different
sets of commands (since we need to erase a block w/a separate invocation
from the real upload).
Update boards.txt to use the new options format, placing the esptool
command as "version" when there is no "erase_flash" or "erase_region" to
be done to keep things simple.
* Move esptool/pyserial to submodules
Since esptool.py and pyserial are coming directly from github repos,
there is no need to include them as a tool in package.json.
* Restore 921K upload opt, silent downgrade to 460k
To enable full backward compatibility, restore the 921k option for
upload speed but silently change it to 460k in the upload.py script.
Add error checking on upload.py
* Keep signing commands in platform.txt on release
The boards packager was deleting what was at the time it was written
unused lines in fht platform.txt file before deploying to Arduino.
One of these lines is now needed for signing to work, so don't delete
it.
Also, explicitly call "python signing.py" because it looks like Arduino
is sanitizing/removing executable bits on files when extracting from
boards manager installations.
Fixes#5483
* Create the build subdir, if needed, for autosign
If the temporary build/core directory isn't available, make it in order
that Updater.cpp will see the generated signing header and not the one
in the main core.
* uart fixes and BW improvements
* uart: read_char straightly use hw buffer
* +attributes for functions called by ISR
* uart: BW improvements
read_char straightly use hw buffer (+ ~10%bw)
read by block (+ ~190%bw) (instead of generic Stream::readBytes)
attributes for functions called by ISR
remove overrun message
remove some ISR flags which were not honoured
* fix merge
* fix buffer overflow
* serial stress test sketch
* astyle
* serial stress example: interactive keyboard, stop reading, overrun
* serial device test: bandwidth & overrun
* update + HardwareSerial::hasError()
* interactive overrun in example
* astyle
* Test using @plerup's SoftwareSerial as submodule (tag 3.4.1)
* update upstream ref (fix warning)
* host mock uart/read(buf,size)
* reset style changes in submodules before style diff
* update build_boards_manager_package.sh for submodules
* trigger CI (removing space)
* cannot reproduce locally the CI issue, setting bash -x option to get live trace
* remove previously added (in this PR) 'set -e' in package builder (passes local tests, not real CI)
script-comment new recipe.hooks.core.prebuild.3 (along with already commented .1 and .2)
moved CI package test to be first on the test list
remove 'set -x', wish me luck
The complete toolchain, including mkspiffs, esptool, C, C++, newlib,
and others (BearSSL excluded) is now built and uploaded with a single
command to ensure repeatability and minimize manual mistakes. All
OSes and architectures are built at a time.
Update to 2.5.0-2 throught the chain.
