* upstream lwIP is now downloaded by a makefile, not subsubmoduled
* lwip2: upstream lwIP not sub-sub-modules anymore
lwip2: Allow IPv4 and IPv6 DNS and SNTP server configured via DHCP to co-exist (patch against upstream)
* lwip2: enable tcp-listen-with-backlog feature
* lwip2 submodule update:
- enable more efficient chksum algorithm thanks to Richard Allen
- enable tcp listener with backlog
* more comments, fix backlog management, fix API
* move default value definition in .cpp
because one must not believe it can be redefined before including WiFiServer.h
* improved backlog handling, it is no more a breaking change
as proposed by https://github.com/esp8266/Arduino/pull/6972#issue-358508056.
@ttytyper 's changes have been incorporate into this PR
The build flag ARDUINO_SONOFF_... should now appear as ARDUINO_ESP8266_SONOFF_...
@ttytyper, @mcspr, and @d-a-v thanks!
* Use a python3 script to call python3
It's odd, but because Windows requires a full Python3 install we must
have an executable called "tools/python3/python3" to use Python3 in the
toolchain.
Before, we simply symlinked to /usr/bin/python3 (for Linux) or
/usr/local/bin/python3 (Mac). Unfortunately, depending on the method of
installation, on MacOS the Python3 executable can be in /usr/bin/python3
instead.
To avoid the entire issue, unify the Mac and Linux python3 placeholders
to use python3 itself to jump to the real executable.
Fixes#6931
* Explicitly remove old symlink to python3
The tar extraction for the updated python3 tarball will fail on systems
that already have a symlink in /tools/python3/python3 because the tar
extractor attempts to open the *target of the symlink* (i.e. the actual
interpreter in /usr/bin or /usr/local/bin).
Add a commented hack to destroy this symlink before expanding the
tarballs, if the file exists. This is safe to do since it will be
overwritten by any extractions of the python3 tarball later in the
process.
Co-authored-by: david gauchard <gauchard@laas.fr>
Only minor updates to headers and no functionality change on the
portions that we use in the ESP8266.
@Teddyz reported random crashes while running against a local MQTT
server and was able to report stack usages of up to 6136 bytes.
Increase the SSL stack to a little more than that, once again.
* Add a CRC32 over progmem and ESP.checkFlashCRC
Automatically embed a CRC32 of the program memory (including bootloader
but excluding any filesystems) in all images in unused space in the
bootloader block.
Add a call, ESP.checkFlashCRC() which returns false if the calculated
CRC doesn't match the one stored in the image (i.e. flash corruption).
Fixes#4165
* Add example that corrupts itself, comments
Show CRC checking catch a 1-bit error in program code by corrupting a
large array, and then return it to clean and verify the CRC matches once
again.
Add comments to the CRC check routine
Clean up pylint complaints on crc32bin.py
* Check linker script for CRC space in bootsector
Add an assertion in the eboot linker file to guarantee that we have at
least 8 bytes of unused space at the end of the boot sector to patch in
the CRC. If not, the eboot link will fail.
* Add note about what to do if CRC check fails
Per discussion with @d-a-v.
When the CRC check fails, you could *try* to do certain things (but may
not succeed since there is known flash corruption at that point). List
a few ideas for application authors.
* Only single, flash/ram friendly crc32() function
* Combine the CRC calc and bin generation in 1 step
Per discussion w/@mcspr, combine the CRC calculation with the binary
generation, removing the additional build step.
* Allow GZIP compressed flash updates
Modified the bootloader to be able to take stored updates in compressed
GZIP format (i.e. the output of "gzip -9 xxx.bin") and decompress them
on-the-fly to their final destination. This can work for apps and for
filesystems (when used with the 2-step update option).
Allow eboot to be built using -Os/2 optimizations by fixing some portions
which failed when any optimizations were used. Add -Wall and use data
and function sections to reduce size. Use -Os to minimize size.
Remove obsolete esptool-ck calls to build a .ROM image, we don't use it.
Move all uninitted variables to RAM from IRAM, allowing 8-bit access.
Hook in @d-a-v and @pfalcon's uzlib port to actually do the
decompression. Do not use any CRC checking which saves space. Since we
have overwritten all of flash by the time we know id the CRC matches,
there's nothing we could have done anyway.
Adjust the Updater class to support GZIP files and not attempt to patch
them.
Bootloader builds to 0xd90 out of 0xfff bytes.
* Add @d-a-v's patch for httpupdate
https://github.com/esp8266/Arduino/pull/6820#pullrequestreview-326541014
* Update uzlib to point to pfalcon++
For now, because there are some self-test failures with @d-a-v's esp8266
branch (whose cool new features we don't actually use in eboot now)
start with pfalcon's 2.9 release and add the 2 patches (clcidx to code
from IRAM/RODATA, and the Windows test file renaming) needed to build
and run successfully.
* Add (c) notice for uzlib to README
* sdk:22x191122
another firmware to test
not default: v2.2.1-119-ga0b1311 (shows as SDK:2.2.2-dev(a58da79) in debug mode)
default unchanged
* and the obj files
The FS uploader plugins need to be updated to use python3 and not
python, or they will fail on Windows (or Linux boxes without an
installed python2 interpreter).
Update the documents to point to the new versions.
* Make upload.py compatible with existing FS upload
PR #6765 introduced an incompatibility with the existing Java uploaders
for SPIFFS and LittleFS, breaking them because of the upload.py
parameter format change to support single stage erase/upload.
Add in patch to silently eat the single --end and to parse the write
address and filename properly as generated by calls from the plugins,
while retaining compatibility with the current IDE changes.
* Clean upload.py, make platform use write_flash
Make upload.py more concise and pythonic.
Use the "write_flash" argument in platform.txt for uploads instead of
assuming anything that is a file is the bin to upload.
* Add boards filter support - allows for the creation of an abridged boards.txt.
Add some of the ITEAD Sonoff boards to boards.txt.py.
Minor reorder of presentation of board menu items, mainly grouped
board Model and module selection to the top.
* Corrected, I think, LED_BUILTIN vs BUILTIN_LED??
* Updated boards.txt
* Added support for DOIT ESP-Mx DevKit (ESP8285) board.
Adjusted wording and fixed side bar formating issue on the
Sonoff description. Also, removed resetmethod menu
and assigned resetmethod of none.
* FW: use NONOS-SDK branch 2.2.x from 2019-10-24 by default
former one is available when using generic board configuration
* tv qvoqve, Platform-ii
* nonos-sdk v2.2.x from 2019-11-05 by default for all boards
(previous ones can be selected with the generic board)
* (w/ fw files)
* 191024 by default, 191105 is an option
* Reset method changes for esptool.py
* Workaround - to be reverted in case esptool do erase/write_flash in one command
* Keep previous resetmethod names and translate to esptool.py options
* Regenerated boards.txt
* adds initial commit to boards.txt before actual edits
* reminder to remove vim .swp file
* Removes spaces and modifies outdated tags for menu `CpuFrequency -> xtal` for example
* Adds SparkFun Bynk Board to boards.txt.py, moves .build flags all together for SparkFun Blynk
* runs boards.txt.py to generate files for pull request - all files have been included as instructed at top of boards.txt file
* Deletes three .orig files generated by boards.txt.py: rst, txt, and json
* Moves boards.txt.py back to /tools directory and attempts to change its mode.
* restore 'x' flag
* sdk: testing branch update (v2.2.x), tidy up fw names in menus, add dates
Former default SDK (22y, renamed to 22x-190703) is still available
Changelog is included in sdk directory
* Keep current FW as default until next release
* update for PIO
This removes definitions relating to the built-in SNTP client that
are LwIP v1 specific. Instead of duplicating these pull in the
LwIP header that correspond to the required functions depending on
the version of the stack being used.
Without this fix calls to sntp_getserver() work but return invalid
data and can lead to stack exhaustion.
Update the NTP-TZ-DST example to use the Arduino sntp.h header
rather than duplicate the conditional checks to use the LwIP header.
Tests:
- Build against a simple SNTP API demonstratin app and all
LwIP configurations. Verify that the app runs for an extended
period and that the expected results are obtained.
Fixes#6590
The ASM block that implements the read-uint32-unaligned returns a
uint32_t. The old code was doing a cast like `(float)(uint32_t ret)
which actually goes and creates a new float of the positive uint value
(approx, of course due to exponent and sign bits) which is not correct.
C and C++ don't have a concise way to convert the bits in a register
from int to float interpretation, so avoid the whole issue by making a
new function which uses the same ASM block as the read-uint32-unaligned,
just make the destination and return values as floats.
* Replace ASM block w/C marco for PSTR
GAS doesn't support the C language idiom of catenating two strings
together with quotes (i.e. "x" "y" === "xy").
Specify the section attribute fully in the section attribute, instead,
to allow this.
* Fix WString optimization
PR #6573 introduced a corner case where a blind String() without any
initialization was in an in invalid state because the buffer and len
would not be updated properly. Concatenating to the empty string could
cause a failure.
Now, set the default state in ::init() to SSO (which is what happened
before when we were using String(char *s="")) and fix the crash.
* Move all PSTRs to own section, allow string dedup
GNU ld can deduplicate strings, and does so for most normal char *s
automatically. However, for PSTRs we were using a unique section per
string *and* the section was not flagges as containing dedupable
0-terminated strings.
Modify the PSTR macro to emit assembly, which lets us set the section
flags required (SM) for the variables, and use inline assembly to get
the asm-block defined address.
Should result in smaller compiled binaries if any strings are
duplicated.
* Give each PSTR its own segment
* Allow disposing of unused strings before merging
Add the "a" section flag to allow the linker to throw away unneeded
strings. Without this flag, the linker will not discard unreferenced
strings and ROMs will increase in size, possibly dramatically.
* Add typedef for putc1, fn_putc1_t.
Replaced relevant usage of `(void *)` with `fn_putc1_t`.
Correct usage of `ets_putc()`, returning 0, in libc_replacement.cpp
This PR assumes PR https://github.com/esp8266/Arduino/pull/6489#issue-315018841 has merged and removes `uart_buff_switch` from `umm_performance.cpp`
Updated method of defining `_rom_putc1` to be more acceptable (I hope) to the new compiler.
* Use PROVIDE to expose ROM function entry point, ets_uart_putc1.
Added comments to ets_putc() and ets_uart_putc1() to explain their
differences. Change prototype of ets_putc() to conform with fp_putc_t.
Updated _isr_safe_printf_P to use new definition, ets_uart_putc1.
In order to give user libs a change to update to the new symbols, re-add
the _SPIFFS_XX symbols to the linker file with a comment that they are
deprecated.
Also add back spiffs_hal_xxx functions, also marked as deprecated.
Fixes#6542
A number of non-genuine boards exist mainly from flea-bay sellers that
use under-sized and/or low quality flash chips which can not handle
a 40MHz FlashFreq properly.
This patch adds slower flash frequencies to the menu for generic ESP boards
so that these cheap knock-offs can be run in a stable manner, hopefully saving
some people a few headaches and keeping these boards out of landfill.
When the FS_END was adjusted to end on a full block (i.e. rounded down)
to avoid filesystem issues, but _FS_end was changed. The EEPROM library
used _FS_end to implicitly calculate the start of the EEPROM data, so
this means after the _FS_end fix, EEPROM data written with prior
releases would "disappear."
Avoid the issue by explicitly calculating the EEPROM start location in
the linker, using the same formula as prior release.
Fixes#6531
Since IRAM is such a precious resource on the ESP8266, dump out its size
(and all other segments) at the end of the build process.
Ex:
Executable segment sizes:
IROM : 338932
IRAM : 27263
DATA : 1476
RODATA : 2896
BSS : 30304
Sketch uses 370567 bytes (35%) of program storage space. Maximum is 1044464 bytes.
Global variables use 34676 bytes (42%) of dynamic memory, leaving 47244 bytes for local variables. Maximum is 81920 bytes.
* Replace the SDK's use of ets_intr_lock/unlock with nestable versions
Testing has shown that there are several paths in the SDK that result in nested
calls to ets_intr_lock() / ets_intr_unlock() which may be a problem.
These functions also do not preserve the enabled interrupt level and may
result in code running with interrupts enabled when that is not intended.
This issue has recently been fixed in the Arduino code by using
xt_rsil() / xt_wsr_ps() but still exists in the Espressif SDK code.
This commit is intended to fix that and should be used in addition to the above.
The maximum nesting I have seen is 2 and lock/unlock calls appear to be balanced.
A max of 7 levels of nesting leaves plenty of room for that to change.
* make ets_intr_lock_stack uint16_t and behave like the original on over/underflow
The PS register is 15 bits, we should store the whole thing as xt_wsr_ps()
writes the whole thing.
Also if there is an underflow, we should make sure interrupts are enabled.
Same goes for overflow making sure interrupts are disabled, although this
is less important.
* Rename ets_intr_(un)lock_nest to ets_intr_(un)lock
This saves having to modify libmain.a, libpp.a and libnet80211.a to use the
nested versions.
Adjusts fix_sdk_libs.sh accordingly.
* Remove ets_intr_(un)lock from the rom .ld as we no longer use them
* ets_post() wrapper to preserve interrupt state
Add a wrapper around the ets_post code in rom to preserve the interrupt enable state.
Rather than modifying the SDK libs, rename ets_post in the .ld file and call the
wrapper "ets_post" to replace it.
As far as I can establish, ets_post is the only rom function in use by our code or
the SDK libs we use that causes calls to ets_intr_(un)lock.
* Add IRAM_ATTR to ets_intr_(un)lock and ets_post wrappers.
* Throw in a few comments and make ets_intr_lock_stack* static.
Build a single INO under a Windows VM on Travis to ensure
that the Win32 toolchain works properly.
In build.py, instead of making a string with spaces and then
splitting on " ", just make the list itself with individual parameters.
This will allow spaces in paths to be supported properly.
* mDNS debug option + AP address is used by default when STA is also present
* mDNS: store network interface, checking it is up
* igmp: force on selected interface (avoid crash *sometimes*)
* fix for all lwip2 ipv4 ipv6 & lwip1
* mdns: IPAddress is not needed to reference associated interface
* mdns: debug: fix print warnings
* emulation: add ets_strncpy
* emulation: truly emulate AddrList (remove fake one)
* Move all scripts and documentation to Python3
Python 2 EOL is Jan 1, 2020. Migrate scripts to run under Python 3.
Under Windows, we're already running Python 3.7, by dumb luck. The
oddness is that the Windows standalone executable for Python 3 is called
"python" whereas under UNIX-like OSes it's called "python3" with
"python" always referring to the Python 2 executable. The ZIP needs to
be updated to include a Python3.exe (copy of Python.exe) so that we can
use the same command lines under Linux and Windows, and to preserve my
sanity.
Fixes#6376
* Add new Windows ZIP with python3.exe file
* Sort options in boards.txt generation for repeatability
The order of the board opts dict changes depending on the Python version
and machine, so sort the options before printing them to get a stable
ordering.
* Re-add Python2 compatibility tweaks
Most scripts can run as Python 2 or Python 3 with minimal changes, so
re-add (and fix, as necessary) compatibility tweaks to the scripts.
Dropped routines form libc.a which are present and usable in ROM:
`ar dv libc.a lib_a-strcmp.o lib_a-strlen.o lib_a-strncmp.o lib_a-strstr.o lib_a-memcmp.o lib_a-memcpy.o lib_a-memmove.o lib_a-memset.o lib_a-bzero.o`
Left strcpy and strncpy in libc.a because they silently support PROGMEM
accesses as required by GCC optimizations.
Saves ~628 bytes in AdvancedWebServer example, from IROM (not IRAM).
Also allows mem* and str* routines to be safely called from inside an
ISR. Prior to this PR, these routines were stored in flash and not
IRAM, so they were technically illegal to call while in an ISR.
Fixes#6430
* Correct critical section with interrupt level preserving and nest support
alternative. Replace ets_intr_lock()/ets_intr_unlock() with uint32_t
oldValue=xt_rsil(3)/xt_wrs(oldValue). Added UMM_CRITICAL_DECL macro to define
storage for current state. Expanded UMM_CRITICAL_... to use unique
identifiers. This helpt facilitate gather function specific timing
information.
Replace printf with something that is ROM or IRAM based so that a printf
that occurs during an ISR malloc/new does not cause a crash. To avoid any
reentry issue it should also avoid doing malloc lib calls.
Refactor realloc to avoid memcpy/memmove while in critical section. This is
only effective when realloc is called with interrupts enabled. The copy
process alone can take over 10us (when copying more than ~498 bytes with a
80MHz CPU clock). It would be good practice for an ISR to avoid realloc.
Note, while doing this might initially sound scary, this appears to be very
stable. It ran on my troublesome sketch for over 3 weeks until I got back from
vacation and flashed an update. Troublesome sketch - runs ESPAsyncTCP, with
modified fauxmo emulation for 10 devices. It receives lost of Network traffic
related to uPnP scans, which includes lots of TCP connects disconnects RSTs
related to uPnP discovery.
I have clocked umm_info critical lock time taking as much as 180us. A common
use for the umm_info call is to get the free heap result. It is common
to try and closely monitor free heap as a method to detect memory leaks.
This may result in frequent calls to umm_info. There has not been a clear
test case that shows an issue yet; however, I and others think they are or
have had crashes related to this.
I have added code that adjusts the running free heap number from _umm_malloc,
_umm_realloc, and _umm_free. Removing the need to do a long interrupts
disabled calculation via _umm_info.
Build optional, min/max time measurements for locks held while in info,
malloc, realloc, and free. Also, maintain a count of how many times each is
called with INTLEVEL set.
* Fixed. travis build complaint.
* Changes for https://github.com/esp8266/Arduino/pull/6274#pullrequestreview-259579883
* Added requested comment and missing comment for UMM_CRITICAL_PERIOD_ANALYZE.
* Updated comments and update xt_rsil()
* Moved xt_rsil&co (pulled in __STRINGIFY) definitions out of
Arduino.h, to cores/esp8266/core_esp8266_features.h
Added esp_get_cycle_count() to core_esp8266_features.h.
Updated umm_malloc and Esp.h to use new defines and location.
* Added "#ifndef CORE_MOCK" around conflicted area.
* Moved performance measurment and ESP specific definitions to
umm_performance.h/cpp. Removed testing asserts.
* Commented out umm analyze. Delay CRITICAL_SECTION_EXIT() in
umm_realloc() to avoid exposing a transient OOM condition to ISR.
* Missed file change. This commit has: Delay CRITICAL_SECTION_EXIT() in
umm_realloc() to avoid exposing a transient OOM condition to ISR.
* 2nd Path. Removed early release of critical section around memmove
to avoid a possible OOM for an ISR.
* improved variable name
* Resolved ISR OOM concern with `_umm_realloc()`
Updated realloc() to do a preliminary free() of unused space,
before performing a critical section exit and memmove.
This change was applied to the current _umm_realloc().
This change should reduce the risk of an ISR getting an
OOM, during a realloc memmove operation.
Added additional stats for verifying correct operation.
* Resolved ISR OOM concern in _umm_realloc()
Updated realloc() to do a preliminary free() of unused space,
before performing a critical section exit and memmove.
This change was applied to the current _umm_realloc().
This change should reduce the risk of an ISR getting an
OOM when interrupting an active realloc memmove operation.
Added additional stats for verifying correct operation.
Updated: for clarity and Travis-CI fail.
* Update to keep access to alternate printf in one file.
* Updated to use ISR safe versions of memmove, memcpy, and memset.
The library versions of memmove, memcpy, and memset were in flash.
Updated to use ROM functions ets_memmove, ets_memcpy, and ets_memset.
Additional note, the library version of memmove does not appear to
have been optimized. It took almost 10x longer than the ROM version.
Renamed printf macro to DBGLOG_FUNCTION and moved to umm_malloc_cfg.h.
Changed printf macro usage to use DBGLOG_FUNCTION.
* Update umm_malloc.cpp
Fix comment
