* upstream lwIP is now downloaded by a makefile, not subsubmoduled
* lwip2: upstream lwIP not sub-sub-modules anymore
lwip2: Allow IPv4 and IPv6 DNS and SNTP server configured via DHCP to co-exist (patch against upstream)
* lwip2: enable tcp-listen-with-backlog feature
* lwip2 submodule update:
- enable more efficient chksum algorithm thanks to Richard Allen
- enable tcp listener with backlog
* more comments, fix backlog management, fix API
* move default value definition in .cpp
because one must not believe it can be redefined before including WiFiServer.h
* improved backlog handling, it is no more a breaking change
Only minor updates to headers and no functionality change on the
portions that we use in the ESP8266.
@Teddyz reported random crashes while running against a local MQTT
server and was able to report stack usages of up to 6136 bytes.
Increase the SSL stack to a little more than that, once again.
* Allow GZIP compressed flash updates
Modified the bootloader to be able to take stored updates in compressed
GZIP format (i.e. the output of "gzip -9 xxx.bin") and decompress them
on-the-fly to their final destination. This can work for apps and for
filesystems (when used with the 2-step update option).
Allow eboot to be built using -Os/2 optimizations by fixing some portions
which failed when any optimizations were used. Add -Wall and use data
and function sections to reduce size. Use -Os to minimize size.
Remove obsolete esptool-ck calls to build a .ROM image, we don't use it.
Move all uninitted variables to RAM from IRAM, allowing 8-bit access.
Hook in @d-a-v and @pfalcon's uzlib port to actually do the
decompression. Do not use any CRC checking which saves space. Since we
have overwritten all of flash by the time we know id the CRC matches,
there's nothing we could have done anyway.
Adjust the Updater class to support GZIP files and not attempt to patch
them.
Bootloader builds to 0xd90 out of 0xfff bytes.
* Add @d-a-v's patch for httpupdate
https://github.com/esp8266/Arduino/pull/6820#pullrequestreview-326541014
* Update uzlib to point to pfalcon++
For now, because there are some self-test failures with @d-a-v's esp8266
branch (whose cool new features we don't actually use in eboot now)
start with pfalcon's 2.9 release and add the 2 patches (clcidx to code
from IRAM/RODATA, and the Windows test file renaming) needed to build
and run successfully.
* Add (c) notice for uzlib to README
* sdk:22x191122
another firmware to test
not default: v2.2.1-119-ga0b1311 (shows as SDK:2.2.2-dev(a58da79) in debug mode)
default unchanged
* and the obj files
* FW: use NONOS-SDK branch 2.2.x from 2019-10-24 by default
former one is available when using generic board configuration
* tv qvoqve, Platform-ii
* nonos-sdk v2.2.x from 2019-11-05 by default for all boards
(previous ones can be selected with the generic board)
* (w/ fw files)
* 191024 by default, 191105 is an option
* adds initial commit to boards.txt before actual edits
* reminder to remove vim .swp file
* Removes spaces and modifies outdated tags for menu `CpuFrequency -> xtal` for example
* Adds SparkFun Bynk Board to boards.txt.py, moves .build flags all together for SparkFun Blynk
* runs boards.txt.py to generate files for pull request - all files have been included as instructed at top of boards.txt file
* Deletes three .orig files generated by boards.txt.py: rst, txt, and json
* Moves boards.txt.py back to /tools directory and attempts to change its mode.
* restore 'x' flag
* sdk: testing branch update (v2.2.x), tidy up fw names in menus, add dates
Former default SDK (22y, renamed to 22x-190703) is still available
Changelog is included in sdk directory
* Keep current FW as default until next release
* update for PIO
This removes definitions relating to the built-in SNTP client that
are LwIP v1 specific. Instead of duplicating these pull in the
LwIP header that correspond to the required functions depending on
the version of the stack being used.
Without this fix calls to sntp_getserver() work but return invalid
data and can lead to stack exhaustion.
Update the NTP-TZ-DST example to use the Arduino sntp.h header
rather than duplicate the conditional checks to use the LwIP header.
Tests:
- Build against a simple SNTP API demonstratin app and all
LwIP configurations. Verify that the app runs for an extended
period and that the expected results are obtained.
Fixes#6590
The ASM block that implements the read-uint32-unaligned returns a
uint32_t. The old code was doing a cast like `(float)(uint32_t ret)
which actually goes and creates a new float of the positive uint value
(approx, of course due to exponent and sign bits) which is not correct.
C and C++ don't have a concise way to convert the bits in a register
from int to float interpretation, so avoid the whole issue by making a
new function which uses the same ASM block as the read-uint32-unaligned,
just make the destination and return values as floats.
* Replace ASM block w/C marco for PSTR
GAS doesn't support the C language idiom of catenating two strings
together with quotes (i.e. "x" "y" === "xy").
Specify the section attribute fully in the section attribute, instead,
to allow this.
* Fix WString optimization
PR #6573 introduced a corner case where a blind String() without any
initialization was in an in invalid state because the buffer and len
would not be updated properly. Concatenating to the empty string could
cause a failure.
Now, set the default state in ::init() to SSO (which is what happened
before when we were using String(char *s="")) and fix the crash.
* Move all PSTRs to own section, allow string dedup
GNU ld can deduplicate strings, and does so for most normal char *s
automatically. However, for PSTRs we were using a unique section per
string *and* the section was not flagges as containing dedupable
0-terminated strings.
Modify the PSTR macro to emit assembly, which lets us set the section
flags required (SM) for the variables, and use inline assembly to get
the asm-block defined address.
Should result in smaller compiled binaries if any strings are
duplicated.
* Give each PSTR its own segment
* Allow disposing of unused strings before merging
Add the "a" section flag to allow the linker to throw away unneeded
strings. Without this flag, the linker will not discard unreferenced
strings and ROMs will increase in size, possibly dramatically.
* Add typedef for putc1, fn_putc1_t.
Replaced relevant usage of `(void *)` with `fn_putc1_t`.
Correct usage of `ets_putc()`, returning 0, in libc_replacement.cpp
This PR assumes PR https://github.com/esp8266/Arduino/pull/6489#issue-315018841 has merged and removes `uart_buff_switch` from `umm_performance.cpp`
Updated method of defining `_rom_putc1` to be more acceptable (I hope) to the new compiler.
* Use PROVIDE to expose ROM function entry point, ets_uart_putc1.
Added comments to ets_putc() and ets_uart_putc1() to explain their
differences. Change prototype of ets_putc() to conform with fp_putc_t.
Updated _isr_safe_printf_P to use new definition, ets_uart_putc1.
In order to give user libs a change to update to the new symbols, re-add
the _SPIFFS_XX symbols to the linker file with a comment that they are
deprecated.
Also add back spiffs_hal_xxx functions, also marked as deprecated.
Fixes#6542
When the FS_END was adjusted to end on a full block (i.e. rounded down)
to avoid filesystem issues, but _FS_end was changed. The EEPROM library
used _FS_end to implicitly calculate the start of the EEPROM data, so
this means after the _FS_end fix, EEPROM data written with prior
releases would "disappear."
Avoid the issue by explicitly calculating the EEPROM start location in
the linker, using the same formula as prior release.
Fixes#6531
* Replace the SDK's use of ets_intr_lock/unlock with nestable versions
Testing has shown that there are several paths in the SDK that result in nested
calls to ets_intr_lock() / ets_intr_unlock() which may be a problem.
These functions also do not preserve the enabled interrupt level and may
result in code running with interrupts enabled when that is not intended.
This issue has recently been fixed in the Arduino code by using
xt_rsil() / xt_wsr_ps() but still exists in the Espressif SDK code.
This commit is intended to fix that and should be used in addition to the above.
The maximum nesting I have seen is 2 and lock/unlock calls appear to be balanced.
A max of 7 levels of nesting leaves plenty of room for that to change.
* make ets_intr_lock_stack uint16_t and behave like the original on over/underflow
The PS register is 15 bits, we should store the whole thing as xt_wsr_ps()
writes the whole thing.
Also if there is an underflow, we should make sure interrupts are enabled.
Same goes for overflow making sure interrupts are disabled, although this
is less important.
* Rename ets_intr_(un)lock_nest to ets_intr_(un)lock
This saves having to modify libmain.a, libpp.a and libnet80211.a to use the
nested versions.
Adjusts fix_sdk_libs.sh accordingly.
* Remove ets_intr_(un)lock from the rom .ld as we no longer use them
* ets_post() wrapper to preserve interrupt state
Add a wrapper around the ets_post code in rom to preserve the interrupt enable state.
Rather than modifying the SDK libs, rename ets_post in the .ld file and call the
wrapper "ets_post" to replace it.
As far as I can establish, ets_post is the only rom function in use by our code or
the SDK libs we use that causes calls to ets_intr_(un)lock.
* Add IRAM_ATTR to ets_intr_(un)lock and ets_post wrappers.
* Throw in a few comments and make ets_intr_lock_stack* static.
Dropped routines form libc.a which are present and usable in ROM:
`ar dv libc.a lib_a-strcmp.o lib_a-strlen.o lib_a-strncmp.o lib_a-strstr.o lib_a-memcmp.o lib_a-memcpy.o lib_a-memmove.o lib_a-memset.o lib_a-bzero.o`
Left strcpy and strncpy in libc.a because they silently support PROGMEM
accesses as required by GCC optimizations.
Saves ~628 bytes in AdvancedWebServer example, from IROM (not IRAM).
Also allows mem* and str* routines to be safely called from inside an
ISR. Prior to this PR, these routines were stored in flash and not
IRAM, so they were technically illegal to call while in an ISR.
Fixes#6430
* Correct critical section with interrupt level preserving and nest support
alternative. Replace ets_intr_lock()/ets_intr_unlock() with uint32_t
oldValue=xt_rsil(3)/xt_wrs(oldValue). Added UMM_CRITICAL_DECL macro to define
storage for current state. Expanded UMM_CRITICAL_... to use unique
identifiers. This helpt facilitate gather function specific timing
information.
Replace printf with something that is ROM or IRAM based so that a printf
that occurs during an ISR malloc/new does not cause a crash. To avoid any
reentry issue it should also avoid doing malloc lib calls.
Refactor realloc to avoid memcpy/memmove while in critical section. This is
only effective when realloc is called with interrupts enabled. The copy
process alone can take over 10us (when copying more than ~498 bytes with a
80MHz CPU clock). It would be good practice for an ISR to avoid realloc.
Note, while doing this might initially sound scary, this appears to be very
stable. It ran on my troublesome sketch for over 3 weeks until I got back from
vacation and flashed an update. Troublesome sketch - runs ESPAsyncTCP, with
modified fauxmo emulation for 10 devices. It receives lost of Network traffic
related to uPnP scans, which includes lots of TCP connects disconnects RSTs
related to uPnP discovery.
I have clocked umm_info critical lock time taking as much as 180us. A common
use for the umm_info call is to get the free heap result. It is common
to try and closely monitor free heap as a method to detect memory leaks.
This may result in frequent calls to umm_info. There has not been a clear
test case that shows an issue yet; however, I and others think they are or
have had crashes related to this.
I have added code that adjusts the running free heap number from _umm_malloc,
_umm_realloc, and _umm_free. Removing the need to do a long interrupts
disabled calculation via _umm_info.
Build optional, min/max time measurements for locks held while in info,
malloc, realloc, and free. Also, maintain a count of how many times each is
called with INTLEVEL set.
* Fixed. travis build complaint.
* Changes for https://github.com/esp8266/Arduino/pull/6274#pullrequestreview-259579883
* Added requested comment and missing comment for UMM_CRITICAL_PERIOD_ANALYZE.
* Updated comments and update xt_rsil()
* Moved xt_rsil&co (pulled in __STRINGIFY) definitions out of
Arduino.h, to cores/esp8266/core_esp8266_features.h
Added esp_get_cycle_count() to core_esp8266_features.h.
Updated umm_malloc and Esp.h to use new defines and location.
* Added "#ifndef CORE_MOCK" around conflicted area.
* Moved performance measurment and ESP specific definitions to
umm_performance.h/cpp. Removed testing asserts.
* Commented out umm analyze. Delay CRITICAL_SECTION_EXIT() in
umm_realloc() to avoid exposing a transient OOM condition to ISR.
* Missed file change. This commit has: Delay CRITICAL_SECTION_EXIT() in
umm_realloc() to avoid exposing a transient OOM condition to ISR.
* 2nd Path. Removed early release of critical section around memmove
to avoid a possible OOM for an ISR.
* improved variable name
* Resolved ISR OOM concern with `_umm_realloc()`
Updated realloc() to do a preliminary free() of unused space,
before performing a critical section exit and memmove.
This change was applied to the current _umm_realloc().
This change should reduce the risk of an ISR getting an
OOM, during a realloc memmove operation.
Added additional stats for verifying correct operation.
* Resolved ISR OOM concern in _umm_realloc()
Updated realloc() to do a preliminary free() of unused space,
before performing a critical section exit and memmove.
This change was applied to the current _umm_realloc().
This change should reduce the risk of an ISR getting an
OOM when interrupting an active realloc memmove operation.
Added additional stats for verifying correct operation.
Updated: for clarity and Travis-CI fail.
* Update to keep access to alternate printf in one file.
* Updated to use ISR safe versions of memmove, memcpy, and memset.
The library versions of memmove, memcpy, and memset were in flash.
Updated to use ROM functions ets_memmove, ets_memcpy, and ets_memset.
Additional note, the library version of memmove does not appear to
have been optimized. It took almost 10x longer than the ROM version.
Renamed printf macro to DBGLOG_FUNCTION and moved to umm_malloc_cfg.h.
Changed printf macro usage to use DBGLOG_FUNCTION.
* Update umm_malloc.cpp
Fix comment
* Upgrade to 2.5.0-4 toolchain w/improved pgm_read_x
Rebuild the entire toolchain (including standard libraries) with the
latest pgm_read_xxx headers included (which fix unaligned dword reads
from progmem and run faster/smaller, and a pgm_read_byte change which
removes an instruction on each read saving flash).
Pull in latest bearssl while we're at it, too, which speeds up EC
handshakes and reduced ROM usage, too.
* Fix C++ exceptions
Exception code now only does 32b aligned reads from progmem to access
the eh_table (some via -mforce-l32, some via hand-inserted pgm_read_x
macros).
Fixes#6151Fixes#6305Fixes#6198
Cleans up all warnings seen w/GCC 9.1 to allow it to track the main
branch more easily until 3.x.
Does not include Ticker.h "fix" of pragmas around a function cast we're
doing that GCC9 doesn't like, that will be addressed separately and
maybe only in the 3.0 branch.
Does not include GDB hook fix, either, because the pragmas required
to disable the GCC9.1 warnings don't exist in 4.8 at all.
* Make SPIFFS be an integer number of blocks
boards.txt.py simply calculated the end and start using flash sizes, but
in cases where an 8K page was used (>512KB SPIFFS), this could leave a
4K half-block left at the end of SPIFFS.
mkspiffs and the SPIFFS code uses integer division to calculate the
maximum block number, so it worked fine in practice and the code simply
ignored the extra, fractional block.
Now actually take block size into account when calculating the end of
SPIFFS, ensuring no fractional blocks are passed in. Does not result in
data loss on pre-existing SPIFFS filesystems.
* Fix the 1m512 case and clean up code
Ensure that no SPIFFS_block in the LD files is modified from the
original to endure correct backwards compatibility
* Factor out common if, clean code
* Make boards.py vars "fs_xx" instead of "spiffs_xx"
Apply most compatible changes needed to get the core compiling under GCC
7.2 to the main gcc 4.8 tree to ease porting for 3.0.0.
Update pgmspace.h with corrected and optimized unaligned pgm_read
macros. Now pgm_read_dword in the unaligned case gives proper results
even if optimization is enabled and is also written in assembly and only
1 instruction longer than the pgm_read_byte macro (which also has been
optimized to reduce 1 instruction). These changes should marginally
shrink code and speed up flash reads accordingly.
The toolchain should/will be rebuilt at a later time with this
optimization to ensure it's used in the libc.a/etc. files.
Undo the BearSSL RODATA->PROGMEM changes because there are some bad
performance regressions in EC server operations which can result in
timeouts and WDTs.
Keep the shrunked bearssl.a library as that is orthogonal to the PROGMEM
changes.
Rewrite all the integer math operations with const input parameters to
use PROGMEM properly (pgm_read_xx or memcpy_P), and move all the EC
order and generators and SHA OIDs to PROGMEM.
This frees around 1.2KB of heap for any SSL applications.
Also delete unneeded objects from the bearssl.a library to shrink the
GIT repo size.
As found by @mhightower83, umm_malloc was placed in flash during the
.c->.cpp conversion because of a missed linker change.
Adjust the link script to the new name .cpp
Move additional constants to flash and use _P/pgm_read routines to
access them. Minimal runtime impact, but remove variables from RODATA
and gives addition 484 bytes of heap to SSL applications.
Fixes#6005
* Add LittleFS as internal flash filesystem
Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem,
https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard
flash utilizing the exact same API as the existing SPIFFS filesystem.
LittleFS is built for low memory systems that are subject to random power
losses, is actively supported by the ARMmbed community, supports directories,
and seems to be much faster in the large-ish read-mostly applications I use.
LittleFS, however, has a larger minimum file allocation unit and does not do
static wear levelling. This means that for systems that need many little
files (<4K), have small SPIFFS areas (64K), or which have a large static
set of files covering the majority of flash coupled with a frequently
updated set of other files, it may not perform as well.
Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch,
use LittleFS.open in place of SPIFFS.open to open files, and everything
else just works thanks to the magic of @igrr's File base class.
**LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS**
Since it is a completely different filesystem, you will need to reformat
your flash (and lose any data therein) to use it. Tools to build the
flash filesystem and upload are at
https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and
https://github.com/earlephilhower/mklittlefs/ . The mklittlefs tool
is installed as part of the Arduino platform installation, automatically.
The included example shows a contrived read-mostly example and
demonstrates how the same calls work on either SPIFFS.* or LittleFS.*
Host tests are also included as part of CI.
Directories are fully supported in LittleFS. This means that LittleFS
will have a slight difference vs. SPIFFS when you use
LittleFS.openDir()/Dir.next(). On SPIFFS dir.next()
will return all filesystem entries, including ones in "subdirs"
(because in SPIFFS there are no subdirs and "/" is the same as any
other character in a filename).
On LittleFS, dir.next() will only return entries in the directory
specified, not subdirs. So to list files in "/subdir/..." you need
to actually openDir("/subdir") and use Dir.next() to parse through
just those elements. The returned filenames also only have the
filename returned, not full paths. So on a FS with "/a/1", "/a/2"
when you do openDir("/a"); dir.next().getName(); you get "1" and "2"
and not "/a/1" and "/a/2" like in SPIFFS. This is consistent with
POSIX ideas about reading directories and more natural for a FS.
Most code will not be affected by this, but if you depend on
openDir/Dir.next() you need to be aware of it.
Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile,
::openNextFile, and ::rewind methods added to Filesystem objects.
Documentation has been updated with this and other LittleFS information.
Subdirectories are made silently when they do not exist when you
try and create a file in a subdir. They are silently removed when
the last file in them is deleted. This is consistent with what
SPIFFS does but is obviously not normal POSIX behavior. Since there
has never been a "FS.mkdir()" method this is the only way to be
compatible with legacy SPIFFS code.
SPIFFS code has been refactored to pull out common flash_hal_* ops
and placed in its own namespace, like LittleFS.
* Fix up merge blank line issue
* Merge in the FSConfig changs from SDFS PR
Enable setConfig for LittleFS as well plys merge the SPIFFS changes
done in the SDFS PR.
* Fix merge errors
* Update to use v2-alpha branch
The V2-alpha branch supports small file optimizations which can help
increase the utilization of flash when small files are prevalent.
It also adds support for metadata, which means we can start adding
things like file creation times, if desired (not yet).
* V2 of littlefs is now in upstream/master
* Update test to support non-creation-ordered files
In a directory, the order in which "readNextFile()" will return a name
is undefined. SPIFFS may return it in order, but LittleFS does not as
of V2. Update the test to look for files by name when doing
readNextFile() testing.
* Fix LittleFS.truncate implementation
* Fix SDFS tests
SDFS, SPIFFS, and LittleFS now all share the same common set of tests,
greatly increasing the SDFS test coverage.
* Update to point to mklittlefs v2
Upgrade mklittlefs to V2 format support
* Remove extra FS::write(const char *s) method
This was removed in #5861 and erroneously re-introduced here.
* Minimize spurious differences from master
* Dramatically reduce memory usage
Reduce the program and read chunk sizes which impacts performance
minimally but reduces per-file RAM usage of 16KB to <1KB.
* Add @d-a-v's host emulation for LittleFS
* Fix SW Serial library version
* Fix free space reporting
Thanks to @TD-er for discovering the issue
* Update littlefs to latest upstream
* Remove sdfat version included by accident
* Update SDFAT to include MOCK changes required
* Update to include SD.h test of file append
* add regular scheduled functions, now also callable on `yield()`
added bool schedule_function_us(std::function<bool(void)> fn, uint32_t repeat_us)
lambda must return true to be not removed from the schedule function list
if repeat_us is 0, then the function is called only once.
Legacy schedule_function() is preserved
This addition allows network drivers like ethernet chips on lwIP to be regularly called
- even if some user code loops on receiving data without getting out from main loop
(callable from yield())
- without the need to call the driver handling function
(transparent)
This may be also applicable with common libraries (mDNS, Webserver, )
The interrupt vectors in IRAM are omitted when there is a PROVIDE
statement in the linker control files when using the PIO method of
-Wl,-T<linkfile>.
Drop the PROVIDES (they're in RAM anyway and not ROM related), and
add the required "-u"s to the PIO build script.
Should have no impact on the Arduino side.
Fixes#6087
Fixes#5996
* Add extensions to probe message for EC, others
probeMFLN was failing on some connection attempts to servers which only
supported EC based ciphers because it did not include the proper TLS
handshake extensions to list what kinds of ECs it supported.
Add those to the probeMFLN ClientHello message to make probes pass.
* Add client.getMFLNStatus method, returns MFLN state
After a connection it is useful to check whether MFLN negotiation
succeeded. getMFLNStatus returns a bool (valid only after
client.connect() succeeds, of course) indicating whether the requested
buffer sizes were negotiated successfully.
* Unaligned access support for pgm_read_word/dword
* Fix pgm_read_ptr_aligned() per #5735
* Allow users to use aligned-only via a #define
Adding -DPGM_READ_UNALIGNED=0 or #define PGM_READ_UNALIGNED 0 will
change the default at compile-time to only aligned (faster, but less
compatible) macro implementations.
Default is still to allow unaligned accesses.
* Split IRAM into 2 linker sections to move std::fcn
Callbacks need to be placed in IRAM when being called from an IRQ (like
the SPISlave callbacks).
This can be done by hacking the std::functional header and making every
single specialization of the template into an IRAM section, which would
take a lot of space for no benefit in the majority of cases.
The alternate is to specify the single instantiation types/operators
required, but the problem is the flash segment matcher would match them
before the IRAM section was begun, and rules for them would just not be
applied.
Get around this by splitting the IRAM section definition into .text and
.text1. This is linker syntactic sugar and does not actually change the
on-chip layout. But it does allow us to put the exception vectors at
the required absolute addresses and add single functions to IRAM.
* Add .text1 segment to space used calculation in IDE
* All functional callers are now placed in IRAM
* Write out the .text1 segment to the BIN
The extra segment name needs to be placed into the output binary as
well, or else only the init code gets stored and none of the real app is
present. This leads to an infinite boot loop.
This change adds in the segment to the generated image.
