Upstream umm_malloc at git hash id 4dac43c3be7a7470dd669323021ba238081da18e
processed all project files with the style program uncrustify.
This PR updates our ported version of umm_malloc processed with "uncrustify".
This should make subsequent merges of upstream into this port easier.
This also makes the style more consistant through umm_malloc.
* StreamConstPtr: prevent from passing a temporary String instance
* unconditionally allow progmem chars
* missing virtual destructor in Stream
(warning: deleting object of abstract class type 'Stream' which has non-virtual destructor will cause undefined behavior [-Wdelete-non-virtual-dtor])
Extracted fix from upstream for umm_blocks() - On allocations that were too
large, umm_blocks() could return an incorrectly truncated value when the result
is cast to uint16_t.
* A clock phase correction in write_start(void)
Some devices require the data wire SDA to be held down at the moment while the clock wire is pulled down too to execute the start condition (e.g. devices using "TinyWires.h" library). This change follows a behaviour of Arduino Wire.h library, where the SCL signal is pulled down in half of period of start condition.
* Formatting of modification restyled
* Removing mathematical operations from delay
* Comments added
esp_yield() now also calls esp_schedule(), original esp_yield() function renamed to esp_suspend().
Don't use delay(0) in the Core internals, libraries and examples. Use yield() when the code is
supposed to be called from CONT, use esp_yield() when the code can be called from either CONT or SYS.
Clean-up esp_yield() and esp_schedule() declarations across the code and use coredecls.h instead.
Implement helper functions for libraries that were previously using esp_yield(), esp_schedule() and
esp_delay() directly to wait for certain SYS context tasks to complete. Correctly use esp_delay()
for timeouts, make sure scheduled functions have a chance to run (e.g. LwIP_Ethernet uses recurrent)
Related issues:
- #6107 - discussion about the esp_yield() and esp_delay() usage in ClientContext
- #6212 - discussion about replacing delay() with a blocking loop
- #6680 - pull request introducing LwIP-based Ethernet
- #7146 - discussion that originated UART code changes
- #7969 - proposal to remove delay(0) from the example code
- #8291 - discussion related to the run_scheduled_recurrent_functions() usage in LwIP Ethernet
- #8317 - yieldUntil() implementation, similar to the esp_delay() overload with a timeout and a 0 interval
These changes are needed to address bugs that can emerge with the improved optimization from the GCC 10.3 compiler.
Updated performance inline functions `mmu_get_uint8()`, ... and `mmu_set_uint8()`, ... to comply with strict-aliasing rules.
Without this change, stale data may be referenced. This issue was revealed in discussions on https://github.com/esp8266/Arduino/issues/8261#issue-963529268
Changes to avoid over-optimization of 32-bit wide transfers from IRAM, turning into 8-bit or 16-bit transfers by the new GCC 10.3 compiler. This has been a reoccurring/tricky problem for me with the new compiler.
So far referencing the 32-bit value loaded by way of an Extended ASM R/W output register has stopped the compiler from optimizing down to an 8-bit or 16-bit transfer.
Example:
```cpp
uint32_t val;
__builtin_memcpy(&val, v32, sizeof(uint32_t));
asm volatile ("" :"+r"(val)); // inject 32-bit dependency
...
```
Updated example `irammem.ino`
* do a simple test of compliance to strict-aliasing rules
* For `mmu_get_uint8()`, added tests to evaluate if 32-bit wide transfers were converted to an 8-bit transfer.
A function called with a flash string, which only has an implementation with `const String&` as argument will be compiled as if it is called with a `String` constructor wrapped around it.
For example this implementation in the .h file:
```c++
bool startsWith(const __FlashStringHelper *prefix) const {
return this->startsWith(String(prefix));
}
```
This is completely useless as the compiler will generate exactly the same code with or without this function implementation in the .h file.
However if we move the implementation to the .cpp file, this conversion to `String` is only added once in the compiled binary.
In my own project I already managed to shrink the largest (ESP32) build by more than 70k in size (!!) by just adding extra function calls with the conversion in the .cpp file.
This PR is just a simple optimisation which already shrinks a very small build of my project by almost 3k in build size. (custom_beta_ESP8266_4M1M PIO env of ESPEasy)
```
Flash: [======== ] 82.5% (used 862137 bytes from 1044464 bytes)
Flash: [======== ] 82.3% (used 859545 bytes from 1044464 bytes)
```
Larger builds may benefit even more.
* Recover the BearSSL crash stack before the Heap is initialized and zeros it.
* Added comments for hwdt_pre_sdk_init()
* Keep Basic ASM version of app_entry_redefinable and removed alternate "C"/Extended ASM version. Update comments.
* Improved example HwdtStackDump to use StackThunk
* Added comments for ets_install_uart_printf and corrected it usage.
* Correct case for hotkey 'p'.
Added conditional build around option 'p' to call stack_thunk_dump_stack
which can only print when debug is enabled.
* BREAKING - "i2s.h" removed, I2S library added
An "i2S.h" header was present in prior cores, but this conflicts with the
Arduino standard I2S.h header for the I2S class under Windows (because of
case insensitivity).
Initial 3.0.0 release has a redirect "i2s.h" file in the cores directory to
redirect to the "I2S.h" file in the library, but under Windows this resulted
in the IDE not building the I2S library and link errors.
Remove the offending header. Code will need to mode to including "I2S.h"
instead (which will include both the Arduino class as well as the low-level
definitions).
Fixes#8107
* Update examples to use proper I2S.h version
fixes for WiFiClient::write(Stream) and Stream transfers
- remove deprecated WiFiClient::write(Stream,size)
- fix and deprecate WiFiClient::write(Stream) to use Stream::sendAll instead of ::sendAvailable
- update ESP8266WebServer::streamFile to use file.sendAll(client) instead of client.write(file)
- remove stream dependence in ClientContext
- Stream::send(): honor timeout in all case, avoid short transfer when output is temporarily full
- example WiFiEcho: show sendAll and sendAvailable
* Disable WiFi at boot by default
* +define WIFI_IS_OFF_AT_BOOT
* remove now useless example
* mv enableWiFiAtBootTime() to core_esp8266_features.h
* sync with master
* per @earlephilhower review: a file was missing
* doc
* WiFi persistence is now false by default
* fix doc
* ditto
* doc: remove sphinx warnings (fix links and formatting)
* fix link name
* fix doc
* legacy: restore persistence
* undeprecate preinit()
* move force modem up to when mode has changed (per @mcspr review)
* do not wake up from sleep when mode if OFF
* fix doc per review
* Hardware WDT Stack Dump
This Sketch demonstrates the use of a tool to print a stack dump
at reboot after a Hardware WDT event.
The module hwdt_app_entry.cpp writes a stack dump to the serial interface
after a Hardware Watchdog Timer has struck and a new boot cycle has begun.
The sketch must properly initialized the Serial port before the crash.
hwdt_app_entry.cpp is the core file that does the work.
* Corrected Style. Improved HWDT reset detectionat boot.
* Style and typos
* Update comments.
* Improvements to reset reason determination.
Improved comments.
Added option to match the UART speed used by the sketch.
Added option to print greeting at the start to indicate the HWDT stack dump code is active.
Isolated logic for handling strings: one assuming they are not inited at
the time the code is running and one that does. The later appears to be the case.
* Style plus
Fix issue with HWDT reason detection when sketch crashes too fast.
Added sample sketch menu option for crashing with a function defined
with a weak attribute via prototype, but never actually defined in
full function form. eg. `void trouble(void){return;}`
* Moved all configuration options to the top.
Adjusted configuration option order for most likely to be used to the top.
Tried to improve comments.
Replace numbers with enum values.
* Removed clutter of having an alternate printing method.
Regular global strings have worked reliably.
Tweaked reset reason detection.
Reordered elements in global structure.
Improve #if test for debug option
Always improving comments.
* Added delays around uart_div_modify. This appeara to resolve the lost
data problem that occurs when printing after a flash upload using esptool.
Curiously, esptool stub also uses similar delays.
Word choices and description improvements.
* Finished TODO looked at assembly of app_entry_redefinable to confirm
no mixed up stack frame was created. Also removed no longer needed
extra level of function calling.
Use existing macros from uart_register,h to handle getting current
UART speed. Added some missing `const`.
* Comment changes.
Added a few newlines to printing.
Decreased the settling delay after the uart_div_modify call.
* Improved comments.
Print caution message when stack integrity checks failed.
* Several corrections to set_uart_speed
Comment improvements
Added missing ";"
* Removed unused include.
Code cleanup.
Comments.
* Now runs from flash before SDK is started.
Cache_Read_Enable working. Free 1K of IRAM and 200 bytes of DRAM.
* Changed ICACHE size from 32K to 16K to avoid conflict with
SDK or core selecting smaller 16K ICACHE.
The Issue: Cache_Read_Enable does not clear the bit field when mapping IRAM to
ICACHE on register 0x3FF00024. Thus, no problem upgrading from 16K to 32K;
however, you cannot downgrade from 32K to 16K. These bits are cleared at boot.
Improved uart data rate change handling.
Update comments.
* Added support to print ThunkStack.
Adjustments to inline asm. Added "memory" when callx0 is used.
* comment cleanup. added missing additional #if defined()
Made structure name unique. Changed HWDT_INFO to HWDR_INFO_S.
* Update style used for structure and typedef to match that used in core
when snake case is used. Moved a constexpr block up a scope so that
some #ifdef debug code compiles again.
* Updated comments
* Corrected new errors from upgrade to GCC 10.1 toolchain related to
constexpr and casting integers to pointers.
Cleared warning for asm.
* Work around divide by 0 HWDT event under toolchain 10.1.
* Changes to move feature into core.
Making ready for selection via tools menu, updated defines to DEBUG_ESP_,,, format.
Added HWDT and HWDT_NO4KEXTRA options to boards.txt.py. These options are selectable
from Arduino IDE 'Tools->Debug Level'
Converted macro names that use to be constexprs to uppercase.
Update comments. Added comments to maintainers anotated by '//C'
Revised example.
* Fix stack character buffer length.
* Updated comment to reflect support via Arduino IDE Tools menu.
* Improve meshing of HWDT and NOEXTRA4K
* Made compatible with `disable_extra4k_at_link_time()` usage.
Changed to strings containing "no4kextra" to "noextra4k" to be consistant with
original usage.
Updated example to provide indications of which build options were used or resulted.
Some comment cleanup.
* CI style
* Adjusted down the ROM Stack space for the extra 4K Heap option.
If too large, a really bad crashes occurs.
Updated the example to start WiFi. This helps double check ROM
Stack space size is not too small at start.
Removed stale comment.
Changed cont stack check functions to make globally available.
* Add replacement aes_unwrap for the debug HWDT option.
Improves the SYS stack space available when using the extra 4K Heap
option in conjunction with HWDT. Replaces the ROM AES buffer at
0x3FFFEA80 with one provided by malloc().
* Update umm_info_safe_printf_P to support default of unaligned PROGMEM strings.
* Improve cont stack trace for yielding case.
Check if cont stack is yielding to SYS, use g_pcont->sp_yield to limit the
amount of the cont stack dumped.
Generalized dev logic path to create a generalized debug function hwdt_pre_sdk_init_icache.
* Added missed update to heap.cpp for change to use PSTR instead of PSTR4
* Updated comments and #if in aes_unwrap.
* Update boards.txt
* wip
* huh, turns out String = 'c' did some weird stuff
* style
* allow "blah" + String, 'c' + String and F("...") + String
also, simplify const char* vs. __FlashStringHelper, and just always use _P functions
* shuffle things into .cpp
* trying to fix arduinojson
based on the implementation, we only need to specify that this symbol is a class
* fix accidental realloc, add test for operator+
basic chaining should work just like with master
comparing std::move() buffers won't work though, because we never allow
anything but `const StringSumHelper&` references
* fixup! fix accidental realloc, add test for operator+
* don't need another branch
* template +=(String / char* / numbers) and +(String, numbers / char*)
* nul after moving (isnt mem always zeroed tho?)
* check if lhs cant keep before switching to rhs
* fix String used to store struct data
`cannot bind bit-field '...' to 'signed char&'
`cannot bind bit-field '...' to 'unssigned char&'
noticed in both tasmota and espeasy, where this generates a webpage
content from some status struct containing bitfields
* style once more
* typo
* recover 444002180bca8e36b3014eaf5ecf5e690837b440
Provides a transparently accessible additional block of RAM of 128K to
8MB by using an external SPI SRAM. This memory is managed using the UMM
memory manager and can be used by the core as if it were internal RAM
(albeit much slower to read or write).
The use case would be for things which are quite large but not
particularly frequently used or compute intensive. For example, the SSL
buffers of 16K++ are a good fit for this, as are the contents of Strings
(both to avoid main heap fragmentation as well as allowing Strings of
>30KB).
A fully associative LRU cache is used to limit the SPI bus bottleneck,
and background writeback is supported.
Uses a define in boards.txt to enable. If this value is not defined,
then the entire VM routines should not be linked in to user apps
so there should be no space penalty w/o it.
UMM `malloc` and `new` are modified to support internal and external
heap regions. By default, everything comes from the standard heap, but
a call to `ESP.setExternalHeap()` before the allocation (followed by a
call to `ESP.resetHeap()` will make the allocation come from external
RAM. See the `virtualmem.ino` example for use.
If there is no external RAM installed, the `setExternalHeap` call is a
no-op.
The String and BearSSL libraries have been modified to use this external
RAM automatically.
Theory of Operation:
The Xtensa core generates a hardware exception (unrelated to C++
exceptions) when an address that's defined as invalid for load or store.
The XTOS ROM routines capture the machine state and call a standard C
exception handler routine (or the default one which resets the system).
We hook into this exception callback and decode the EXCVADDR (the
address being accessed) and use the exception PC to read out the
faulting instruction. We decode that instruction and simulate it's
behavior (i.e. either loading or storing some data to a
register/external memory) and then return to the calling application.
We use the hardware SPI interface to talk to an external SRAM/PSRAM,
and implement a simple cache to minimize the amount of times we need
to go out over the (slow) SPI bus. The SPI is set up in a DIO mode
which uses no more pins than normal SPI, but provides for ~2X faster
transfers. SIO mode is also supported.
NOTE: This works fine for processor accesses, but cannot be used by
any of the peripherals' DMA. For that, we'd need a real MMU.
Hardware Configuration (only use 3.3V compatible SRAMs!):
SPI byte-addressible SRAM/PSRAM: 23LC1024 or smaller
CS -> GPIO15
SCK -> GPIO14
MOSI -> GPIO13
MISO -> GPIO12
(note these are GPIO numbers, not the Arduino Dxx pin names. Refer
to your ESP8266 board schematic for the mapping of GPIO to pin.)
Higher density PSRAM (ESP-PSRAM64H/etc.) should work as well, but
I'm still waiting on my chips so haven't done any testing. Biggest
concern is their command set and functionality in DIO mode. If DIO
mode isn't supported, then a fallback to SIO is possible.
This PR originated with code from @pvvx's esp8266web server at
https://github.com/pvvx/esp8266web (licensed in the public domain)
but doesn't resemble it much any more. Thanks, @pvvx!
Keep a list of the last 8 lines in RAM (~.5KB of RAM) and use that to
speed up things like memcpys and other operations where the source and
destination addresses are inside VM RAM.
A custom set of SPI routines is used in the VM system for speed and code
size (and because the core cannot be dependent on a library).
Because UMM manages RAM in 8 byte chunks, attempting to manage the
entire 1M available space on a 1M PSRAM causes the block IDs to
overflow, crashing things at some point. Limit the UMM allocation to
only 256K in this case. The remaining space can manually be assigned to
buffers/etc. managed by the application, not malloc()/free().
