Remove nonos-sdk-v2.2.0-28-g89920dc aka 3v0, since we have a real v3
Remove 3.0.0...3.0.4 to reduce overhead in maintaining a bunch or binary patches
Update our docs and menu opts to mention 'experimental' status
Old versions still remains in git history, so anyone wanting to play around with 3.0.x could still make use of that work
## WPA2 Enterprise connections
References - merged PRs:
* https://github.com/esp8266/Arduino/pull/8529
* https://github.com/esp8266/Arduino/pull/8566 - these occurred with connect/disconnect with WPA-Enterprise
* https://github.com/esp8266/Arduino/pull/8736#issue-1470774550
The NON-OS SDK 3.0.x has breaking changes to the [`pvPortMalloc`](bf890b22e5/include/mem.h (L42)) function. They added a new `bool` argument for selecting a heap.
```cpp
void *pvPortMalloc (size_t sz, const char *, unsigned, bool);
```
To avoid breaking the build, I added a new thin wrapper function `sdk3_pvPortMalloc` to `heap.cpp`.
Edited new SDK LIBs to call `pvPortMalloc`'s replacement `sdk3_pvPortMalloc`.
They also added `pvPortZallocIram` and `pvPortCallocIram`, which are not a problem to support. Support added to `heap.cpp`.
Issues with WPA2 Enterprise in new SDKs:
* v3.0.0 and v3.0.1 - have the same memory leak and duplicate free bugs from before
* v3.0.2 through v3.0.5 - have the same memory leak; however, _no_ duplicate free crash.
* memory leak can be seen by cycling through setup, connect, disconnect, and clear setup - repeatedly.
Updated `wpa2_eap_patch.cpp` and binary patch scripts to handle v3.0.0 through v3.0.5.
Patched SDKs v3.0.0 through v3.0.5
## Duplicate Non-32-bit exception handler
Issue: At v3.0.0 and above `libmain.a` supplies a built-in exception handler (`load_non_32_wide_handler`) for non-32-bit access. Our non-32-bit access handler (`non32xfer_exception_handler`) overrides it.
Solution: Add "weak" attribute to symbol `load_non_32_wide_handler`. Adjust the build to default to the SDK's built-in non-32-bit handler. If there is a need to use our non-32-bit handler, make the selection from the Arduino IDE Tools menu `Non-32-Bit Access: "Byte/Word access to IRAM/PROGMEM (very slow)"`.
With SDKs v3.0.0 and above a "non-32-bit exception handler" is always present.
Docs doesn't include any reference to include <ESP8266httpUpdate.h> library in Advanced OTA Updater section. It took me some time to find out while reading the docs
Api for saving heap when Client class is used by a Server (WiFiServer class): Client = Server.available().
Suppose the local end is the server and the remote end is the client, we will deal with heap memory at the local end.
When the local application (server) decides to close an active connection with a remote end it issues an Client.stop.
The stop() function calls the close() function of ClientContext class which in turn calls tcp_close.
The connexion is closed by tcp_close and the protocol control block (pcb) can be put in the following states depending on the requests sent by the remote: CLOSING, FIN_WAIT_1 and FIN_WAIT_2. In theses states pcbs are not freed, then consume some memory heap.
If an acknowledgment from the remote end is received, the pcb enter in TIME_WAIT state for some minutes but pcbs in TIME_WAIT state are not freed. Then consume some heap memory.
TIME_WAIT pcbs are automatically freed after some minutes or can be freed for instance issuing an tcp_kill_timewait()
in the local application which will free the oldest pcb in TIME_WAIT state.
If the connection is first closed from the remote end (the client), the local end (server) receive a connection termination request. It then acknowledge it and enter in CLOSE_WAIT state waiting for a connection termination request from the local application.
It then send a termination request and enter in LAST_ACK state until it receive an acknowledgment from the remote end.
After receiving the acknowledgment it enter in ClOSED state and the local pcb is freed leaving some room in the heap memory.
To summarize, when a connexion termination request is send by one end (remote or local), the local pcb is not freed immediatly.
This pcb can be in the following states: FIN_WAIT_1, FIN_WAIT_2, CLOSING, TIME_WAIT, CLOSE_WAIT, LAST_ACK.
As a consequence, some old pcbs from old closed connections are still consuming heap memory.
The local application can call tcp_kill_timewait hoping it will free some TIME_WAIT state pcbs. But if the server receive frequent connections requests and close them after sending whatever it has to send, there may be zero pcbs in TIME_WAIT state among all previously closed connections.
In case of insufficient memory to accept a new connection, lwip has developped a strategy: it successively tries to kill the oldest pcb in TIME_WAIT state, or in LAST_ACK state or in CLOSING state or the oldest active connection with lower priority than the new one.
As a matter of fact this "urgent" strategy is deployed only when very few heap memory remain available (less than some kb). In case of success, Client.available returns a valid Client but the local application will crash when sending or receiving data from the client (Client.read ou readuntil or available) because this need more heap memory and just some kb were freed in lwip to allocate the new pcb structure ans start the new connection.
The propose API is intended to avoid this drawback by calling the abort function of ClientContext which in turn calls tcp_abort which calls tcp_abandon. The connection is aborted and notified to the client with a RESET and the pcb and ressources associated are immediately released increasing the available heap memory.
* Add support to specify global build defines and options
A script manages the use of a file with a unique name, like
`SketchName.ino.globals.h`, in the Sketch source directory to provide compiler
command-line options (build options) and sketch global defines. The build
option data is encapsulated in a unique "C" comment block and extracted into
the build tree during prebuild.
* Applied os.path.normpath() liberally to input arguments. Fixes windows file path issue.
Improved helpful message for adding embedded build options.
* doubleup '\'
* Added context help for build option support
* expunged sketchbook global
added workaround for aggressive caching
* inital pass at searching for and reading preferences.txt
* Correct Windows path for preferences.txt
Added portable path for preferences.txt
Expanded file timestamp granularity
Improved error message printing for Arduino IDE 2.0 RC4
* Improved portable path and various Windows paths to preferences.txt
* Add cleanup logic and identify 1st run after IDE restart
* text corrections
* Create mkbuildoptglobals.py
When global header file does not exist, this print makes it easier for user to create the header file by providing its name and documentation pointer.
* build.opt heads up to user
Compiler command line changes from build.opt are shown to user
* Updated text
* oops
* Expanded comment and made print help consistent
* Improve handling stderr/stdout with "no verbose output"
Grouped helpful info to print at the end.
Added missing return value.
* Correct timestamp on CommonHFile.h
More improvements to printing
Updated docs.
* Added command-line parser
Support hints for compiler.cache_core. For use when Arduino IDE uses
command-line options that override compiler.cache_core.
Removed overuse of ()
Improve FAQ entry
* Fix script failure under windows
Rely on argpaser for checking that all arguments are present.
Removed redundant argument check in main().
Added '--debug' option and print_dbg method.
Rethink failures on overrides. Remove well know path fallbacks,
error exit when override file is missing.
In well-known path search for preferences.txt, do not assume true.
Make failure to find an error exit event.
When Windows has two preferences.txt files and they have different
values for caching and globals.h is used, error exit. It is not
possible to know from the script which is being used.
* Use quotes on build.opt
Update comment
Include the @ within the expantion string use quotes around file name.
Update doc example to remind and use quotes.
* Update CI for build option and global support
Added "mkbuildoptglobals.extra_flags=--cache_core" to platform.loca.txt
Update "-ide-version=10802" this version number indicates aggressive caching support
Added example to test global .h support
* Add debug prints
Added --debug to CI - this needs to be removed later
Tweaks to touch...
* Give each build VM a unique build.tmp space
* Corrected style on example
temp CI changes
debug crud
Added --ci switch
* Removed CI debug crud
run_CI_locall.sh works fine locally. Hosted Multi-VM CI fails
to work with 'aggressive caching' workaround method.
Add #if defined(CORE_MOCK) to failing example.
* Try HOST_MOCK
* CI adjustments
mkbuildoptglobals.py is optimized around the Arduino IDE 1.x
behaviour. One way the CI differs from the Arduino IDE is in the
handling of core and caching core. With the Arduino IDE, each sketch
has a private copy of core and contributes to a core cache. With the
CI, there is one shared copy of core for all sketches. When global
options are used, the shared copy of core and cache are removed before
and after the build.
* Doc update
per #8421 and #8475
after checking HW, suggest to check SW as well by at least by enabling some or all compiler warnings
(and also note of the IDE weirdest defaults causing issues we expected to stay solved)
* WiFiServer - 'rename' available() to accept()
* use server.accept() instead of available()
* WiFiServer.accept() and ArduinoWiFiServer class doc update
* ESP8266WebServer - drop current HC_WAIT_READ connection sooner when another has data
Safari sometimes opens two connections when loading a page and only
sends a request over the second one, resulting in a 5 second wait
(HTTP_MAX_DATA_WAIT) before the request is processed. This commit
drops the current connection after 30ms (HTTP_MAX_DATA_AVAILABLE_WAIT)
when there is a new connection with data available or the buffer of
pending TCP clients is full (currently 5).
* Create pins_arduino.h
* Update boards.txt.py
* Update boards.txt
* done a "boards.txt.py --allgen"
* Removed the definition of LED_BUILTIN from variants/wifi_kit_8/pins_arduino.h, because this board has no build in lED
* wifi: remove pseudo-modes for shutdown
make shutdown and resumeFromShutdown public
removes extra code from the mode handler and include method description
in the docs
* typo
* dup
* typos
* reference only shutdown() & resumeFromShutdown()
prefer to have some specific function, don't simply chain into sleep
update examples and docs
* safeguard raw sdk config usage
* Disable WiFi at boot by default
* +define WIFI_IS_OFF_AT_BOOT
* remove now useless example
* mv enableWiFiAtBootTime() to core_esp8266_features.h
* sync with master
* per @earlephilhower review: a file was missing
* doc
* WiFi persistence is now false by default
* fix doc
* ditto
* doc: remove sphinx warnings (fix links and formatting)
* fix link name
* fix doc
* legacy: restore persistence
* undeprecate preinit()
* move force modem up to when mode has changed (per @mcspr review)
* do not wake up from sleep when mode if OFF
* fix doc per review
* Add setSSLVersion call to SSL object
Allow users to only allow specific TLS versions for connections with an
additional call in their app, similar to the setCiphers call.
Fixes#7918
* Add SSL level options to WiFiServerSecure
Improve resume speed by passing in last known BSSID
Provide a simpler example for WIFI_SHUTDOWN/WIFI_RESUME
Add documentation for WIFI_SHUTDOWN and WIFI_RESUME.
* allow to set pin to OUTPUT_OPEN_DRAIN in analogWrite
* remove parameter with default value
* Update core_esp8266_wiring_pwm.cpp
* update documentation accordingly
* WiFiServerSecure: Cache the SSL sessions
* Add SSL session caching to HTTPS server examples
* Document server SSL session caching
* Fix an incomplete sentence in the documentation
* Document BearSSL::Session
* Use the number of sessions instead of the buffer size in ServerSessions' constructors
* PoC cache configuration control
Expaned boards.txt.py to allow new MMU options and create revised .ld's
Updated eboot to pass 48K IRAM segments.
Added Cache_Read_Enable intercept to modify call for 16K ICACHE
Update platform.txt to pass new mmu options through to compiler and linker preprocessor.
Added quick example: esp8266/MMU48K
* Style corrections
Added MMU_ qualifier to new defines.
Moved changes into their own file.
Don't know how to fix platformio issue.
* Added detailed description for Cache_Read_Enable.
Updated tools/sizes.py to report correct IRAM size and indicate ICACHE size.
Merged in earlephilhower's work on unaligned exception. Refactored and added
support for store operations and changed the name to be more closely aligned
with its function. Improved crash reporting path.
* Style and MMU_SEC_HEAP corrections.
* Improved asm register usage.
Added some inline functions to aid in byte and short access to iRAM.
* only byte read has been tested
Updated .ld file to work better with platform.io; however, I am still
missing some steps, so platformio will still fail.
* Interesting glitch in boards.txt after github merge. A new board in
master was missing new additions added by boards.txt.py in the PR.
Which the CI flags when it rebuilds boards.txt.
* Support for 2nd Heap, excess IRAM, through umm_malloc.
Adapted changes to umm_malloc, Esp.cpp, StackThunk.cpp,
WiFiClientSecureBearSSL.cpp, and virtualmem.ino to irammem.ino from
@earlephilhower PR #6994.
Reworked umm_malloc to use context pointers instead of copy context.
umm_malloc now supports allocations from IRAM. Added class
HeapSelectIram, ... to aid in selecting alternate heaps,
modeled after class InterruptLock.
Restrict alloc request from ISRs to DRAM.
Never ending improvements to debug printing.
Sec Heap option now pulls in free IRAM left over in the 1st 32K block.
Managed through umm_malloc with HeapSelectIram.
Updated examples.
* Post push CI cleanup.
* Cleanup part II
* Cleanup part III
* Updates to support platformio, maybe.
* Added exception C wrapper replacement.
* CI Cleanup
* CI Cleanup II
Don't know what to do with platformio it doesn't like my .S file.
ifdef out USE_ISR_SAFE_EXC_WRAPPER to block the new assemlby module
from building on platformio only.
* Changes to exc-c-wrapper-handler.S to assemble under platformio.
* For platformio, Correction to toolchain-xtensa include path.
@mcspr, Thankyou!
* Temporarily added --print-memory-usage to ld parameters for cross-checking IRAM size.
* undo change to platform.txt
* correct merge conflict. take 1
* Fixed #if... for building umm_get_oom_count. It was not building when UMM_STATS_FULL was used.
* Commented out XMC support. Compatibility issues with PoC when using 16K ICACHE.
* Corrected size.py, DRAM bracketing changed to not include ICACHE with DRAM total.
* Added additional _context for support of use of UMM_INLINE_METRICS.
Corrected some UMM_POSION missed edits.
* Changes to clear errors and warnings from toolchain 10.1
Several fixes and improvements to example MMU48K.
With the improved optimization in toolchain 10.1 The example divide by 0
exception was failing with a HWDT event instead of its exception handler.
The compiler saw the obscured divide by 0 and replaced it with a break point.
* Isolated incompatable definitions related to _xtos_set_exception_handler.
GDBSTUB definitions are different from the BootROM's.
* Update tools/platformio-build.py
Co-authored-by: Max Prokhorov <prokhorov.max@outlook.com>
* Requested changes
Changed mmu related usages of ETS_... defines to DBG_MMU_...
Cleanup in example MMU48K.ino. Removed stale memory reference macro
and mmu_status print statement. Cleanup printf '\n' to be '\r\n'.
Improved issolation of development debug prints from the rest of the debug prints.
* Corrected comment. And added missing include.
* Improve comment.
* style and comment correction
* Added draft mmu.rst file and updated index.
Updated example HeapMetric.ino to also illustrate use of IRAM
Improved comments in exc-c-wrapper-handler.S. Added insurance IRQ disable.
* Updated mmu.rst
Improved function name uniqueness for is_iram, is_dram, and is_icache by
adding prefix mmu_. Also, made them available outside of a debug build.
Made pointer precision width more specific.
Made some of the static inline functions in mmu_irm.h safe for ISRs by
setting then for always inline.
* Add a default MMU_IRAM_SIZE value for a new CI test to pass.
Extended use 'umm_heap_context_t *_context' argument in ..._core functions
and expanded its usage to reduce unnecessary repeated calls to
umm_info(NULL, false), also removed recursion from umm_info(NULL, true).
Fixed stack buffer length in umm_info_safe_printf_P and heap.cpp.
Added example for creating an IRAM reserve section.
Updated mmu.rst. Grammar and spelling corrections.
* CI appeasement
* CI appeasement with comment correction.
* Ensure SYS always runs with DRAM Heap selected.
* Add/move heap stack overflow/underflow check to Esp.cpp where the event was discarded.
* Improved comment clarity of purpose for IramReserve.ino. Clean up MMU48K.ino
* Added missing #include
* Corrected usage of warning
* CI appeasement and use #message not #pragma message
* Updated git version of eboot.elf to match build version.
Good test catch.
* Remove conditional build option USE_ISR_SAFE_EXC_WRAPPER, always install.
Use the replacement wrapper on non32xfer_exception_handler install.
Added comments to code describing some exception handling issues.
* Updated mmu.rst
* Expanded and clarified comments.
Limited access to some detailed typdefs/prototypes to .cpp
modules, to avoid future build conflicts.
Completed TODO for verifing that the "C" structure struct __exception_frame
matches the ASM version.
Fixed some typo's, code rot, and added some more cases in examaple irammem.ino.
Refactored a little and reordered printing to ease comparison between methods.
Corrected `#ifdef __cplusplus` coverage area. Cleaned up `extern "C" ...` usage.
Fixes issues with including mmu_iram.h or esp8266_undocumented.h in .c files.
* Style fixes and more cleanup
* Style fix
* Remove unnessasary IRAM_ATTR from install_non32xfer_exception_handler
Some comment tuning.
In the context of _xtos_set_exception_handler and the functions it registers,
changed to type int for exception cause type. This is also the type used by gdbstub
and some other Xtensa files I found.
