* Reslove flash address issues with SDK v3.0.0
Fix EEPROM vs RF_CAL flash address conflict. The EEPROM address and
RF_CAL address were the same.
Add support for Flash size: "Mapping defined by Hardware and Sketch"
Change at_partition_table static from dynamic to static.
* Cleanup and improve comments
* Improve flash size and partition error reporting/indication
Changed set_pll() to mmu_set_pll() and made available for debug builds
and other settings where required.
Provide more checks and feedback in the debug builds and
trim code for production.
* Now supports FLASH_MAP_SUPPORT with SDKs v3.0
RF_CAL and system_parameter always occupy the last 5
sectors of flash memory.
* cleanup and refactoring
comment cleanup
* Add more build issolation when including flash_hal.h
* Improve details for autoconfig fail.
* requested changes
* stack decoder
* +x
* cut here
* last alloc explain, line breaks
* capture
* print ctx line
* ...and dont ignore sp
* non-hyphenated arg for elf, toolchain path to bin/
either for when tools are already in PATH
or, using `pio pkg exec --package toolchain-xtensa python decoder.py ...`
(where package is a full version spec for pio registry)
* Don't return `true` with WiFiClientSecureBearSSL::connected() when disconnected
Apply the same condition as with normal WiFiClient - we are not connected
when it's not possible to both write and read.
Implement separate methods for actual connection status and the internal
ssl engine status and update methods that were previously using available()
for this purpose
Update examples to check available() when the intent is to only read the
data and not interact with the client in any other way. Also, use connect()
as a way to notify errors, no need to check things twice
## WPA2 Enterprise connections
References - merged PRs:
* https://github.com/esp8266/Arduino/pull/8529
* https://github.com/esp8266/Arduino/pull/8566 - these occurred with connect/disconnect with WPA-Enterprise
* https://github.com/esp8266/Arduino/pull/8736#issue-1470774550
The NON-OS SDK 3.0.x has breaking changes to the [`pvPortMalloc`](bf890b22e5/include/mem.h (L42)) function. They added a new `bool` argument for selecting a heap.
```cpp
void *pvPortMalloc (size_t sz, const char *, unsigned, bool);
```
To avoid breaking the build, I added a new thin wrapper function `sdk3_pvPortMalloc` to `heap.cpp`.
Edited new SDK LIBs to call `pvPortMalloc`'s replacement `sdk3_pvPortMalloc`.
They also added `pvPortZallocIram` and `pvPortCallocIram`, which are not a problem to support. Support added to `heap.cpp`.
Issues with WPA2 Enterprise in new SDKs:
* v3.0.0 and v3.0.1 - have the same memory leak and duplicate free bugs from before
* v3.0.2 through v3.0.5 - have the same memory leak; however, _no_ duplicate free crash.
* memory leak can be seen by cycling through setup, connect, disconnect, and clear setup - repeatedly.
Updated `wpa2_eap_patch.cpp` and binary patch scripts to handle v3.0.0 through v3.0.5.
Patched SDKs v3.0.0 through v3.0.5
## Duplicate Non-32-bit exception handler
Issue: At v3.0.0 and above `libmain.a` supplies a built-in exception handler (`load_non_32_wide_handler`) for non-32-bit access. Our non-32-bit access handler (`non32xfer_exception_handler`) overrides it.
Solution: Add "weak" attribute to symbol `load_non_32_wide_handler`. Adjust the build to default to the SDK's built-in non-32-bit handler. If there is a need to use our non-32-bit handler, make the selection from the Arduino IDE Tools menu `Non-32-Bit Access: "Byte/Word access to IRAM/PROGMEM (very slow)"`.
With SDKs v3.0.0 and above a "non-32-bit exception handler" is always present.
The current implementation of the Servo lib always resets the position when detaching.
In AVR Servo, this isn't the case, instead, it doesn't move the servo but leaves it as it was before getting detached.
Docs doesn't include any reference to include <ESP8266httpUpdate.h> library in Advanced OTA Updater section. It took me some time to find out while reading the docs
Previously, File::readString used a C-style string as an intermediate
buffer via the String += operator. This treats a NUL byte as a
terminator, making this function work incorrectly if the File contains
binary data.
This commit switches the function to use String::concat, which doesn't
treat NUL bytes any differently (and is a bit faster, because it doesn't
need to use strlen).
Shortening up the default hard-coded 8000 ms timeout, will save energy when module can't reach its update server, especially on battery powered projects, and otherwise provide adaptability to the local wlan. The ESPhttpUpdate object is auto-created by #including ESP8266HTTPUpdate. ESP8266HTTPUpdate also provides a constructor with param httpClientTimeout, but to use it required destroying the auto ESPhttpUpdate object, or creating a secondESPhttpUpdate(timeout) object.
Api for saving heap when Client class is used by a Server (WiFiServer class): Client = Server.available().
Suppose the local end is the server and the remote end is the client, we will deal with heap memory at the local end.
When the local application (server) decides to close an active connection with a remote end it issues an Client.stop.
The stop() function calls the close() function of ClientContext class which in turn calls tcp_close.
The connexion is closed by tcp_close and the protocol control block (pcb) can be put in the following states depending on the requests sent by the remote: CLOSING, FIN_WAIT_1 and FIN_WAIT_2. In theses states pcbs are not freed, then consume some memory heap.
If an acknowledgment from the remote end is received, the pcb enter in TIME_WAIT state for some minutes but pcbs in TIME_WAIT state are not freed. Then consume some heap memory.
TIME_WAIT pcbs are automatically freed after some minutes or can be freed for instance issuing an tcp_kill_timewait()
in the local application which will free the oldest pcb in TIME_WAIT state.
If the connection is first closed from the remote end (the client), the local end (server) receive a connection termination request. It then acknowledge it and enter in CLOSE_WAIT state waiting for a connection termination request from the local application.
It then send a termination request and enter in LAST_ACK state until it receive an acknowledgment from the remote end.
After receiving the acknowledgment it enter in ClOSED state and the local pcb is freed leaving some room in the heap memory.
To summarize, when a connexion termination request is send by one end (remote or local), the local pcb is not freed immediatly.
This pcb can be in the following states: FIN_WAIT_1, FIN_WAIT_2, CLOSING, TIME_WAIT, CLOSE_WAIT, LAST_ACK.
As a consequence, some old pcbs from old closed connections are still consuming heap memory.
The local application can call tcp_kill_timewait hoping it will free some TIME_WAIT state pcbs. But if the server receive frequent connections requests and close them after sending whatever it has to send, there may be zero pcbs in TIME_WAIT state among all previously closed connections.
In case of insufficient memory to accept a new connection, lwip has developped a strategy: it successively tries to kill the oldest pcb in TIME_WAIT state, or in LAST_ACK state or in CLOSING state or the oldest active connection with lower priority than the new one.
As a matter of fact this "urgent" strategy is deployed only when very few heap memory remain available (less than some kb). In case of success, Client.available returns a valid Client but the local application will crash when sending or receiving data from the client (Client.read ou readuntil or available) because this need more heap memory and just some kb were freed in lwip to allocate the new pcb structure ans start the new connection.
The propose API is intended to avoid this drawback by calling the abort function of ClientContext which in turn calls tcp_abort which calls tcp_abandon. The connection is aborted and notified to the client with a RESET and the pcb and ressources associated are immediately released increasing the available heap memory.
* Enable SPI_CS_SETUP for early ICACHE use
The SPI_CS_SETUP parameter has been observed set by RTOS SDK and NONOS SDK
as part of flash init/configuration. It may be necessary for some flash
chips to perform correctly with ICACHE hardware access. Turning on and
leaving it on should be okay.
* Cleanup comment
* Change umm_init to default to IRAM
Some flash chips (PUYA) have some unknown requirements for running with
early `Cache_Read_Enable`. They work fine after the SDK is started.
For now, change umm_init to default to IRAM.
Define UMM_INIT_USE_ICACHE to move to ICACHE and free up IRAM.
Added some experimental code that may indirectly support PUYA.
Note, until this issue is resolved, that HWDT Stack Dump is not
going to work with PUYA flash.
* typo
* Finalize fix for PUYA flash and preSDK use of Cache_Read_Enable.
This resolves the exception 0 issue with PUYA flash when using
flash/ICACHE for umm_init and/or using HWDT Stack Dump.
Adds max duration check. In case it is over SDK limit, enable 'repeat'ing timer with a duration proportional to the original one and count until it executes N times, only then run the callback.
Code with durations less than that executes as usual. Original proposal was to not create anything or create some kind of error state... which seems counter-productive to not help out with this pretty solvable use-case.
Additional updates, while refactoring the class
- Stronger types for internal time management using `std::chrono::duration`. Works the same, `std::chrono::duration` handles seconds <-> milliseconds conversion, and we don't have to remember the time type in each method. (...and even allow `once()` and `attach` as overloads instead of the current `_ms`-suffix, in a future update)
- `::detach()` when timer finishes. Fixes (unintentional?) side-effect that we remain `::active()`. Plus, this destroys any lambda-bound variables that will persist with the Ticker object. And, since we can't re-arm with the existing function (`Ticker::attach_ms(uint32_t just_the_time)` and etc.)
- `std::variant` aka union for internal callback storage (kind-of similar to #6918). Instead of having two separate code paths, **always** attach our static function and dispatch using type info. Also helps with the issue described above, since it will call `std::function` dtor when ptr + arg is attached instead of doing nothing.
- smarter copy and move, detaching existing timer on assignment and detaching the moved-in timer object in both ctor and assignment. Copying or moving a running timer no longer blindly copies `_timer` pointer, allowing to disarm the original one. Since we are a simple wrapper around `os_timer_t`, just do the simpler thing (and not re-schedule the callback, try to store original times, etc. polledTimeout already does it and is copyable)
* sprinkle IPAddress(...).isSet() across our loops to avoid polling on a
stopped interface. status callback and netif_is_up **does not
guarantee and we could use the interface**!
* register *one* status callback per instance, e.g. when begin() is called
multiple times (also notice a subtle issue with schedule function when
instance is delete'ed)
* consistent LwipIntf callback signature. no need for rvalue, just pass
stdfunc as-is and let the compiler figure it out
- split workflows into separate files to trigger by path
this should help out documentation and boards / eboot / pkg files
updates, since those *wont* trigger usual build stuff anymore
- build*.sh whatever merged into just common.sh and build.sh
trigger different parity builds, mod % rem and allow to set .ino list
through the environment variable
- removes unnecessary temporary files, try to use more pipes
move remaining ones into cache dir instead of PWD
- remove legacy TRAVIS env vars, use ESP8266_ARDUINO prefix for config
- remove Windows path workarounds
- hardware/ and ide/ directories are set through envionment
do not force specific paths, simplify builds on local machine
- sketch list is set through environment. expicit paths for Windows and
macOS builders. platformio also gets a real shuffled list instead of
mod and rem magic numbers
- detect root of the repo through git cli, not base{name,dir} or relative paths
Wire everything that relies on stack smashing detection to call
`__stack_chk_fail()` (aka what libssp / ssp / stack-protector uses)
Expose it in our debugging header
Rename overflow -> smashing, as these are different things we are trying
to detect (meaning, that we check for things writing there, not some
kind of `alloca` issue or the way `-fstack-check` would have worked)
ref. #8666
`-fstack-protector` continues to work as it always did
CONT replaces `abort()`, also moves its check to the loop wrapper to
avoid dumping otherwise useless SYS context memory
StackThunk replaces a similar `abort()` call
Create dedicated function for xPortGetFreeHeapSize() that only reports on DRAM.
NONOS SDK API system_get_free_heap_size() relies on xPortGetFreeHeapSize() for the free Heap size.
Possible breaking change for multiple Heap Sketches calling system_get_free_heap_size(); it will now always report free DRAM Heap size.
Update and export umm_free_heap_size_lw() to report the free Heap size of the current Heap.
Updated ESP.getFreeHeap() to use umm_free_heap_size_lw().
Updated build options to supply exported umm_free_heap_size_lw() via either UMM_STATS or UMM_INFO.
Improved build option support via the SketchName.ino.globals.h method for Heap options: UMM_INFO, UMM_INLINE_METRICS, UMM_STATS, UMM_STATS_FULL, UMM_BEST_FIT, and UMM_FIRST_FIT. While uncommon to change from the defaults, you can review umm_malloc_cfgport.h for more details, which may help reduce your Sketch's size in dire situations. Assuming you are willing to give up some functionality.
For debugging UMM_STATS_FULL can offer additional stats, like Heap low water mark (umm_free_heap_size_min()).
The NONOS SDK's `struct bss_info` in `user_interface.h` has grown since the
beginning of this project. The additional elements are not accessible.
Add a method for R/O access to full `struct bss_info`.
See #7965 (comment)
Amends #8507
I took the liberty to also do some refactoring; specifically, fixing signed vs. unsigned mismatch in len, using pointer object vs. the original manual malloc & free, try to have named constants for certain addresses and lengths, plus localize printing of u8 arrays.
The suggested test to have a 'dummy' verifier works just fine. (...how it actually works and gets the hash to compare with is a whole other question, though)
Another issue noticed while testing, in the underlying bearssl api there's an actual limit for hash length.
6105635531/inc/bearssl_rsa.h (L257)
* make WiFi/Ethernet interface compatible with Arduino Ethernet API
provide some minimaly adapted examples from legacy
* move ethernet compat globals to EthernetCompat.h
* LegacyEthernet: add UDP example
* adjust comments
Co-authored-by: Max Prokhorov <prokhorov.max@outlook.com>
* Board ID as build flag
When using IDE or `arduino-cli board list`, show full board name and FQBN
```
$ arduino-cli.exe board list
Port Protocol Type Board Name FQBN Core
xxx.xxx.x.xxx network Network Port LOLIN(WEMOS) D1 R2 & mini esp8266com:esp8266:d1_mini esp8266com:esp8266
```
Also add `description` field to the mDNS response containing the
original `{build.board}` string value.
resolve#7759
* does not work with recent ide (?)
* fixup! does not work with recent ide (?)
Separate page handling logic and the actual writing. Make sure we place both unaligned src and dest into a buffer.
Fixes edge-case introduced for SPIFFS that exclusively works through unaligned flash write function.
This copies the behaviour of official RTOS port, but does not change the original spi_flash_write.
