arduino/esp8266
1
0
Fork 0
mirror of https://github.com/esp8266/Arduino.git synced 2025-06-06 05:21:22 +03:00
Code

Update core with upstream umm_malloc (#6438)

Browse Source 
* WIP: Initial overlay of upstream version of umm_malloc over
our current version w/o any edits.

Using files from:
  https://github.com/rhempel/umm_malloc/tree/master/src
  https://github.com/rhempel/c-helper-macros/tree/develop

The following file remanents from our old version were removed:
  umm_malloc.cpp, umm_performance.h, umm_performance.cpp, and umm_stats.h

This is intended to be a "compare to" for changes made to
upstream version.

* WIP: upstream version of umm_malloc customized for Arduino ESP8266 Core
See comments at the top of umm_malloc.cpp for a summary of changes so far.

Added support for integrity check and poison check to refactored heap.cpp.

* Resoved Travis CI - issue

* Corrected #ifdef's for UMM_STATS/UMM_STATS_FULL

* Corrected UMM_REALLOC_DEFRAG option to be more like the original umm_realloc.
Updated some comments and changed method of defining _rom_putc1.

* Updated UMM_POISON_CHECK_LITE - Expanded the poison check on the current
allocation to include its nearest allocated neighbors in the heap.
That is to say, if the nearest neighbor is free, then the next one over
is checked. And this is done in both directions.
umm_malloc() will also checks the neighbors of the selected
allocation before use.

Merged umm_malloc.c into umm_malloc.cpp.

Added additional stats for debug build.

On going comment cleanup.

Removed #if 0 block from isr_safe_printf.

Corrected mistyped UMM_POISON_CHECK UMM_CHECK_POISON.
Corrected other conditional build issues.

* Updated printing in heap.cpp to behave the way os_printf did.

Updated printing in heap.cpp to behave the way os_printf would have with regards
to system_set_os_print. ie. setDebugOutput enable/disables printing

Expanded #ifdef fenceing of .c files to prevent Arduino IDE from building them
outside of the .cpp file they are included in.

Corrected more conditional building issues.

Remaining concern - umm_info(0, true) printing - does it need an ISR safe method,
all printing is done from within a critical sectionn with rsil(15)?
Or should be have the print option disabled for non-debug builds.
ie. Debug port: "Disabled"

* Merged latest, 2019-09-07, from upstream umm_malloc

* R.Hempel 2019-09-07 - Separate the malloc() and free() functionality into
*                        wrappers that use critical section protection macros
*                        and static core functions that assume they are
*                        running in a protected con text. Thanks @devyte

* Resolved issue where OOM selected w/ Debug port: "Disabled" caused hang.

With OOM selected w/ Debug port: "Disabled" you can now use
Serial.setDebugOutput(true); to enable OOM message displaying.
This is the behavior previously available with os_printf.

commit rev 1

* Fixed realloc OOM bug introduced due to confusing code, added comment for future.

umm_info(NULL, true) is now treated as a debug option that is enabled by
Arduino IDE->tools `Debug port: "Serial"`, etc. It now prints when
DEBUG_ESP_PORT or UMM_INFO_PRINT are defined.

Cleanup comments and some conditionals.

* Updated to track PRs for various print definitions. Cleaned up comments

* Added missing "#ifdef __cplusplus"

* Reorganize local enhancements
More comment cleanups
Added Status TODOs marks for upstream

* Print notice message for the need of -DUMM_INFO_PRINT

* This updates the heap management library, umm_malloc, to the current upstream
version at https://github.com/rhempel/umm_malloc. Some reorganizing and new code
was needed to use the new version.

This is a list of note worthy changes:

UMM_POISON - now has a lite option as well as the previous intensive check
option. The code for running the full poison test at the call of the various
alloc functions was removed in the upstream version. In this port the missing
code was added to heap.cpp and umm_local.cpp.
* UMM_POISON - appears to have been partially changed to UMM_POISON_CHECK,
  I treat it as depricated and used UMM_POISON_CHECK, when needed.
  However, the Arduino Core's references to UMM_POISON were replaced with
  UMM_POISON_CHECK_LITE.
* UMM_POISON_CHECK_LITE - Less intense, it just checks poison on active
  neighboring allocations.
* UMM_POISON_CHECK - Full heap intensive check of poison

UMM_INFO_PRINT - This new define makes building UMM_INFO with printing
capability, optional. When umm_info(NULL, true) is used to print a debug view of
heap information to the debug port, it has to walk the heap and print out
information, while in a critical section. This requires that the print function
be able to print w/o doing malloc calls and from an IRQ disabled context. It
also requires more IRAM to handle printing. Without this define
`umm_info(NULL, true)` will not print.
* UMM_INFO_PRINT is enabled as part of selecting `Debug port: "Serial" or
* "Serial1"`. To make available all the time use '-D UMM_INFO_PRINT`.

A cautionary note, on the use of UMM_INTEGRITY_CHECK, UMM_POISON_CHECK, and
UMM_INFO_PRINT. All of these run with IRQs disabled, for periods that can go
into 100's of us. With umm_info(NULL, true) that may go into seconds, depending
on the serial interface speed and the number of memory allocations present. Use
UMM_INTEGRITY_CHECK, UMM_POISON_CHECK, and UMM_INFO_PRINT sparingly.
If you want to see numbers for the disabled time, explore using
UMM_CRITICAL_METRICS in umm_malloc_cfg.h.

* This updates the heap management library, umm_malloc, to the current upstream
version at https://github.com/rhempel/umm_malloc. Some reorganizing and new code
was needed to use the new version.

This is a list of note worthy changes:

UMM_POISON - now has a lite option as well as the previous intensive check
option. The code for running the full poison test at the call of the various
alloc functions was removed in the upstream version. In this port the missing
code was added to heap.cpp and umm_local.cpp.
* UMM_POISON - appears to have been partially changed to UMM_POISON_CHECK,
  I treat it as depricated and used UMM_POISON_CHECK, when needed.
  However, the Arduino Core's references to UMM_POISON were replaced with
  UMM_POISON_CHECK_LITE.
* UMM_POISON_CHECK_LITE - Less intense, it just checks poison on active
  neighboring allocations.
* UMM_POISON_CHECK - Full heap intensive check of poison

UMM_INFO_PRINT - This new define makes building UMM_INFO with printing
capability, optional. When umm_info(NULL, true) is used to print a debug view of
heap information to the debug port, it has to walk the heap and print out
information, while in a critical section. This requires that the print function
be able to print w/o doing malloc calls and from an IRQ disabled context. It
also requires more IRAM to handle printing. Without this define
`umm_info(NULL, true)` will not print.
* UMM_INFO_PRINT is enabled as part of selecting `Debug port: "Serial" or
* "Serial1"`. To make available all the time use '-D UMM_INFO_PRINT`.

A cautionary note, on the use of UMM_INTEGRITY_CHECK, UMM_POISON_CHECK, and
UMM_INFO_PRINT. All of these run with IRQs disabled, for periods that can go
into 100's of us. With umm_info(NULL, true) that may go into seconds, depending
on the serial interface speed and the number of memory allocations present. Use
UMM_INTEGRITY_CHECK, UMM_POISON_CHECK, and UMM_INFO_PRINT sparingly.
If you want to see numbers for the disabled time, explore using
UMM_CRITICAL_METRICS in umm_malloc_cfg.h.

* Updated upstream TODO

* Changes on printing to comply with new understanding of SPI bus availability
from an ISR vs forground with INTLEVEL 15.

* Removed no longer needed files.

* OOM build option now saves file and line number for post mortem reporting.

* Missed change.

* Consolated upstream integration comments into Notes.h. This helps
keep these comments separated from the upstream code and to avoid
getting lost. I'll expand this as I think of and remember more.

* Comment updates.

* Notes update.

* Changes to heap.cpp -
Refactored macro PTR_CHECK__LOG_LAST_FAIL.
Replaced macros __umm_... with uppercase versions.
Corrected misplaced POISON_CHECK__PANIC_FL macro in realloc.

Added debug free options for OOM and Poison to umm_malloc_cfg.h

Updated Notes.h.

Lines that start with "//C" are meant to flag reviewer's attention.

* Corrected Full Poison Check's placement and documented the
thinking behind its placement and Integrity Checks placement
with relation to the *alloc function. Just so I don't forget
again :/

* Fixed typos
This commit is contained in:
M Hightower 2019-10-28 23:42:30 -07:00 committed by Develo
parent 6e51ef0cc8
commit 7a43092df0
18 changed files with 2507 additions and 1972 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
cores/esp8266/core_esp8266_postmortem.cpp
View File

@ -62,6 +62,10 @@ static int s_user_reset_reason = REASON_DEFAULT_RST;
// From UMM, the last caller of a malloc/realloc/calloc which failed:
extern void *umm_last_fail_alloc_addr;
extern int umm_last_fail_alloc_size;
#if defined(DEBUG_ESP_OOM)
extern const char *umm_last_fail_alloc_file;
extern int umm_last_fail_alloc_line;
#endif
static void raise_exception() __attribute__((noreturn));
@ -181,7 +185,13 @@ void __wrap_system_restart_local() {
// Use cap-X formatting to ensure the standard EspExceptionDecoder doesn't match the address
if (umm_last_fail_alloc_addr) {
ets_printf_P(PSTR("\nlast failed alloc call: %08X(%d)\n"), (uint32_t)umm_last_fail_alloc_addr, umm_last_fail_alloc_size);
#if defined(DEBUG_ESP_OOM)
ets_printf_P(PSTR("\nlast failed alloc call: %08X(%d)@%S:%d\n"),
(uint32_t)umm_last_fail_alloc_addr, umm_last_fail_alloc_size,
umm_last_fail_alloc_file, umm_last_fail_alloc_line);
#else
ets_printf_P(PSTR("\nlast failed alloc call: %08X(%d)\n"), (uint32_t)umm_last_fail_alloc_addr, umm_last_fail_alloc_size);
#endif
}
custom_crash_callback( &rst_info, sp_dump + offset, stack_end );

399
cores/esp8266/heap.cpp
View File

@ -7,38 +7,134 @@
#include "umm_malloc/umm_malloc.h"
#include <c_types.h>
#include <sys/reent.h>
#include <user_interface.h>
extern "C" {
#if defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE)
#define UMM_MALLOC(s) umm_poison_malloc(s)
#define UMM_CALLOC(n,s) umm_poison_calloc(n,s)
#define UMM_REALLOC_FL(p,s,f,l) umm_poison_realloc_fl(p,s,f,l)
#define UMM_FREE_FL(p,f,l) umm_poison_free_fl(p,f,l)
#undef realloc
#undef free
#elif defined(DEBUG_ESP_OOM)
#define UMM_MALLOC(s) umm_malloc(s)
#define UMM_CALLOC(n,s) umm_calloc(n,s)
#define UMM_REALLOC_FL(p,s,f,l) umm_realloc(p,s)
#define UMM_FREE_FL(p,f,l) umm_free(p)
#undef realloc
#undef free
#else // ! UMM_POISON_CHECK && ! DEBUG_ESP_OOM
#define UMM_MALLOC(s) malloc(s)
#define UMM_CALLOC(n,s) calloc(n,s)
#define UMM_REALLOC_FL(p,s,f,l) realloc(p,s)
#define UMM_FREE_FL(p,f,l) free(p)
#endif
#if defined(UMM_POISON_CHECK)
#define POISON_CHECK__ABORT() \
do { \
if ( ! POISON_CHECK() ) \
abort(); \
} while(0)
#define POISON_CHECK__PANIC_FL(file, line) \
do { \
if ( ! POISON_CHECK() ) \
__panic_func(file, line, ""); \
} while(0)
#else // No full heap poison checking.
#define POISON_CHECK__ABORT() do {} while(0)
#define POISON_CHECK__PANIC_FL(file, line) do { (void)file; (void)line; } while(0)
#endif
// Debugging helper, last allocation which returned NULL
void *umm_last_fail_alloc_addr = NULL;
int umm_last_fail_alloc_size = 0;
#if defined(DEBUG_ESP_OOM)
const char *umm_last_fail_alloc_file = NULL;
int umm_last_fail_alloc_line = 0;
#endif
#ifdef UMM_INTEGRITY_CHECK
#define INTEGRITY_CHECK__ABORT() \
do { \
if ( ! INTEGRITY_CHECK() ) \
abort(); \
} while(0)
#define INTEGRITY_CHECK__PANIC_FL(file, line) \
do { \
if ( ! INTEGRITY_CHECK() ) \
__panic_func(file, line, ""); \
} while(0)
#else // ! UMM_INTEGRITY_CHECK
#define INTEGRITY_CHECK__ABORT() do {} while(0)
#define INTEGRITY_CHECK__PANIC_FL(file, line) do { (void)file; (void)line; } while(0)
#endif // UMM_INTEGRITY_CHECK
#if defined(DEBUG_ESP_OOM)
#define PTR_CHECK__LOG_LAST_FAIL_FL(p, s, f, l) \
if(0 != (s) && 0 == p)\
{\
umm_last_fail_alloc_addr = __builtin_return_address(0);\
umm_last_fail_alloc_size = s;\
umm_last_fail_alloc_file = f;\
umm_last_fail_alloc_line = l;\
}
#define PTR_CHECK__LOG_LAST_FAIL(p, s) \
if(0 != (s) && 0 == p)\
{\
umm_last_fail_alloc_addr = __builtin_return_address(0);\
umm_last_fail_alloc_size = s;\
umm_last_fail_alloc_file = NULL;\
umm_last_fail_alloc_line = 0;\
}
#else
#define PTR_CHECK__LOG_LAST_FAIL_FL(p, s, f, l) \
(void)f;\
(void)l;\
if(0 != (s) && 0 == p)\
{\
umm_last_fail_alloc_addr = __builtin_return_address(0);\
umm_last_fail_alloc_size = s;\
}
#define PTR_CHECK__LOG_LAST_FAIL(p, s) \
if(0 != (s) && 0 == p)\
{\
umm_last_fail_alloc_addr = __builtin_return_address(0);\
umm_last_fail_alloc_size = s;\
}
#endif
void* _malloc_r(struct _reent* unused, size_t size)
{
(void) unused;
void *ret = malloc(size);
if (0 != size && 0 == ret) {
umm_last_fail_alloc_addr = __builtin_return_address(0);
umm_last_fail_alloc_size = size;
}
PTR_CHECK__LOG_LAST_FAIL(ret, size);
return ret;
}
void _free_r(struct _reent* unused, void* ptr)
{
(void) unused;
return free(ptr);
free(ptr);
}
void* _realloc_r(struct _reent* unused, void* ptr, size_t size)
{
(void) unused;
void *ret = realloc(ptr, size);
if (0 != size && 0 == ret) {
umm_last_fail_alloc_addr = __builtin_return_address(0);
umm_last_fail_alloc_size = size;
}
PTR_CHECK__LOG_LAST_FAIL(ret, size);
return ret;
}
@ -46,140 +142,169 @@ void* _calloc_r(struct _reent* unused, size_t count, size_t size)
{
(void) unused;
void *ret = calloc(count, size);
if (0 != (count * size) && 0 == ret) {
umm_last_fail_alloc_addr = __builtin_return_address(0);
umm_last_fail_alloc_size = count * size;
PTR_CHECK__LOG_LAST_FAIL(ret, count * size);
return ret;
}
#ifdef DEBUG_ESP_OOM
#undef malloc
#undef calloc
#undef realloc
#define DEBUG_HEAP_PRINTF ets_uart_printf
void ICACHE_RAM_ATTR print_loc(size_t size, const char* file, int line)
{
(void)size;
(void)line;
if (system_get_os_print()) {
DEBUG_HEAP_PRINTF(":oom(%d)@", (int)size);
bool inISR = ETS_INTR_WITHINISR();
if (inISR && (uint32_t)file >= 0x40200000) {
DEBUG_HEAP_PRINTF("File: %p", file);
} else if (!inISR && (uint32_t)file >= 0x40200000) {
char buf[ets_strlen(file)] __attribute__ ((aligned(4)));
ets_strcpy(buf, file);
DEBUG_HEAP_PRINTF(buf);
} else {
DEBUG_HEAP_PRINTF(file);
}
DEBUG_HEAP_PRINTF(":%d\n", line);
}
}
void ICACHE_RAM_ATTR print_oom_size(size_t size)
{
(void)size;
if (system_get_os_print()) {
DEBUG_HEAP_PRINTF(":oom(%d)@?\n", (int)size);
}
}
#define OOM_CHECK__PRINT_OOM(p, s) if (!p) print_oom_size(s)
#define OOM_CHECK__PRINT_LOC(p, s, f, l) if (!p) print_loc(s, f, l)
#else // ! DEBUG_ESP_OOM
#if 1
//C - to be discussed - is this what you want?
//C Skip OOM logging of last fail for malloc/... and pvPort... .
//C It cost 64 more bytes of IRAM to turn on. And was not previously enabled.
#undef PTR_CHECK__LOG_LAST_FAIL_FL
#define PTR_CHECK__LOG_LAST_FAIL_FL(p, s, f, l)
#undef PTR_CHECK__LOG_LAST_FAIL
#define PTR_CHECK__LOG_LAST_FAIL(p, s)
#endif
#define OOM_CHECK__PRINT_OOM(p, s)
#define OOM_CHECK__PRINT_LOC(p, s, f, l)
#endif
#if defined(DEBUG_ESP_OOM) || defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE) || defined(UMM_INTEGRITY_CHECK)
/*
The thinking behind the ordering of Integrity Check, Full Poison Check, and
the specific *alloc function.
1. Integrity Check - verifies the heap management information is not corrupt.
This allows any other testing, that walks the heap, to run safely.
2. Place Full Poison Check before or after a specific *alloc function?
a. After, when the *alloc function operates on an existing allocation.
b. Before, when the *alloc function creates a new, not modified, allocation.
In a free() or realloc() call, the focus is on their allocation. It is
checked 1st and reported on 1ST if an error exists. Full Posion Check is
done after.
For malloc(), calloc(), and zalloc() Full Posion Check is done 1st since
these functions do not modify an existing allocation.
*/
void* ICACHE_RAM_ATTR malloc(size_t size)
{
INTEGRITY_CHECK__ABORT();
POISON_CHECK__ABORT();
void* ret = UMM_MALLOC(size);
PTR_CHECK__LOG_LAST_FAIL(ret, size);
OOM_CHECK__PRINT_OOM(ret, size);
return ret;
}
void* ICACHE_RAM_ATTR calloc(size_t count, size_t size)
{
INTEGRITY_CHECK__ABORT();
POISON_CHECK__ABORT();
void* ret = UMM_CALLOC(count, size);
PTR_CHECK__LOG_LAST_FAIL(ret, count * size);
OOM_CHECK__PRINT_OOM(ret, size);
return ret;
}
void* ICACHE_RAM_ATTR realloc(void* ptr, size_t size)
{
INTEGRITY_CHECK__ABORT();
void* ret = UMM_REALLOC_FL(ptr, size, NULL, 0);
POISON_CHECK__ABORT();
PTR_CHECK__LOG_LAST_FAIL(ret, size);
OOM_CHECK__PRINT_OOM(ret, size);
return ret;
}
void ICACHE_RAM_ATTR free(void* p)
{
INTEGRITY_CHECK__ABORT();
UMM_FREE_FL(p, NULL, 0);
POISON_CHECK__ABORT();
}
#endif
void* ICACHE_RAM_ATTR pvPortMalloc(size_t size, const char* file, int line)
{
INTEGRITY_CHECK__PANIC_FL(file, line);
POISON_CHECK__PANIC_FL(file, line);
void* ret = UMM_MALLOC(size);
PTR_CHECK__LOG_LAST_FAIL_FL(ret, size, file, line);
OOM_CHECK__PRINT_LOC(ret, size, file, line);
return ret;
}
void* ICACHE_RAM_ATTR pvPortCalloc(size_t count, size_t size, const char* file, int line)
{
INTEGRITY_CHECK__PANIC_FL(file, line);
POISON_CHECK__PANIC_FL(file, line);
void* ret = UMM_CALLOC(count, size);
PTR_CHECK__LOG_LAST_FAIL_FL(ret, count * size, file, line);
OOM_CHECK__PRINT_LOC(ret, size, file, line);
return ret;
}
void* ICACHE_RAM_ATTR pvPortRealloc(void *ptr, size_t size, const char* file, int line)
{
INTEGRITY_CHECK__PANIC_FL(file, line);
void* ret = UMM_REALLOC_FL(ptr, size, file, line);
POISON_CHECK__PANIC_FL(file, line);
PTR_CHECK__LOG_LAST_FAIL_FL(ret, size, file, line);
OOM_CHECK__PRINT_LOC(ret, size, file, line);
return ret;
}
void* ICACHE_RAM_ATTR pvPortZalloc(size_t size, const char* file, int line)
{
INTEGRITY_CHECK__PANIC_FL(file, line);
POISON_CHECK__PANIC_FL(file, line);
void* ret = UMM_CALLOC(1, size);
PTR_CHECK__LOG_LAST_FAIL_FL(ret, size, file, line);
OOM_CHECK__PRINT_LOC(ret, size, file, line);
return ret;
}
void ICACHE_RAM_ATTR vPortFree(void *ptr, const char* file, int line)
{
(void) file;
(void) line;
free(ptr);
}
#ifdef DEBUG_ESP_OOM
void* ICACHE_RAM_ATTR pvPortMalloc(size_t size, const char* file, int line)
{
return malloc_loc(size, file, line);
}
void* ICACHE_RAM_ATTR pvPortCalloc(size_t count, size_t size, const char* file, int line)
{
return calloc_loc(count, size, file, line);
}
void* ICACHE_RAM_ATTR pvPortRealloc(void *ptr, size_t size, const char* file, int line)
{
return realloc_loc(ptr, size, file, line);
}
void* ICACHE_RAM_ATTR pvPortZalloc(size_t size, const char* file, int line)
{
return calloc_loc(1, size, file, line);
}
#undef malloc
#undef calloc
#undef realloc
static const char oom_fmt[] PROGMEM STORE_ATTR = ":oom(%d)@?\n";
static const char oom_fmt_1[] PROGMEM STORE_ATTR = ":oom(%d)@";
static const char oom_fmt_2[] PROGMEM STORE_ATTR = ":%d\n";
void* malloc (size_t s)
{
void* ret = umm_malloc(s);
if (!ret)
os_printf(oom_fmt, (int)s);
return ret;
}
void* calloc (size_t n, size_t s)
{
void* ret = umm_calloc(n, s);
if (!ret)
os_printf(oom_fmt, (int)s);
return ret;
}
void* realloc (void* p, size_t s)
{
void* ret = umm_realloc(p, s);
if (!ret)
os_printf(oom_fmt, (int)s);
return ret;
}
void print_loc (size_t s, const char* file, int line)
{
os_printf(oom_fmt_1, (int)s);
os_printf(file);
os_printf(oom_fmt_2, line);
}
void* malloc_loc (size_t s, const char* file, int line)
{
void* ret = umm_malloc(s);
if (!ret)
print_loc(s, file, line);
return ret;
}
void* calloc_loc (size_t n, size_t s, const char* file, int line)
{
void* ret = umm_calloc(n, s);
if (!ret)
print_loc(s, file, line);
return ret;
}
void* realloc_loc (void* p, size_t s, const char* file, int line)
{
void* ret = umm_realloc(p, s);
if (!ret)
print_loc(s, file, line);
return ret;
}
#else
void* ICACHE_RAM_ATTR pvPortMalloc(size_t size, const char* file, int line)
{
(void) file;
(void) line;
return malloc(size);
}
void* ICACHE_RAM_ATTR pvPortCalloc(size_t count, size_t size, const char* file, int line)
{
(void) file;
(void) line;
return calloc(count, size);
}
void* ICACHE_RAM_ATTR pvPortRealloc(void *ptr, size_t size, const char* file, int line)
{
(void) file;
(void) line;
return realloc(ptr, size);
}
void* ICACHE_RAM_ATTR pvPortZalloc(size_t size, const char* file, int line)
{
(void) file;
(void) line;
return calloc(1, size);
}
#endif // !defined(DEBUG_ESP_OOM)
size_t xPortGetFreeHeapSize(void)
{
return umm_free_heap_size();
INTEGRITY_CHECK__PANIC_FL(file, line);
UMM_FREE_FL(ptr, file, line);
POISON_CHECK__PANIC_FL(file, line);
}
size_t ICACHE_RAM_ATTR xPortWantedSizeAlign(size_t size)

251
cores/esp8266/umm_malloc/Notes.h Normal file
View File

@ -0,0 +1,251 @@
#if 0
/*
* This .h is nothing but comments about thoughts and observations made while
* updating the Arduino ESP8266 Core, with the new upstream umm_malloc. It is
* added as a .h so that it does not get lost and to avoid cluttering up the
* code with a huge block comment.
PR text description:
upstream version of `umm_malloc` customized for Arduino ESP8266 Core
This updates the heap management library, umm_malloc, to the current upstream
version at https://github.com/rhempel/umm_malloc. Some reorganizing and new code
was needed to use the new version.
This is a list of noteworthy changes:
UMM_POISON - now has a lite option as well as the previous intensive check
option. The code for running the full poison test at the call of the various
alloc functions was removed in the upstream version. In this port, the missing
code was added to heap.cpp and umm_local.cpp.
* UMM_POISON - appears to have been partially changed to UMM_POISON_CHECK,
I treat it as deprecated and used UMM_POISON_CHECK when needed.
However, the Arduino Core's references to UMM_POISON were replaced with
UMM_POISON_CHECK_LITE.
* UMM_POISON_CHECK_LITE - Less intense, it just checks poison on active
neighboring allocations.
* UMM_POISON_CHECK - Full heap intensive check of poison
A cautionary note, on the use of UMM_INTEGRITY_CHECK and UMM_POISON_CHECK, and
umm_info(). All of these run with IRQs disabled, for periods that can go into
100's of us. With umm_info(NULL, true) that may go into seconds, depending on
the serial interface speed and the number of memory allocations present. Use
UMM_INTEGRITY_CHECK, UMM_POISON_CHECK, and umm_info() sparingly. If you want to
see numbers for the disabled time, explore using UMM_CRITICAL_METRICS in
umm_malloc_cfg.h.
===============================================================================
New upstream umm_malloc feature delta's from the old umm_malloc we were using:
umm_posion check for a given *alloc - failure - no longer panics.
option to run full poison check at each *alloc call, not present
option to run full interity check at each *alloc call, not present
upstream code does not call panic from poison_check_block.
Defragmenting effect of realloc is gone. It now minimizes copy. This
may have been an accident during code cleanup.
In one form or another these features have been restored in the
reintegration of the upstream umm_malloc into the Arduino ESP8266 Core.
===============================================================================
A list of changes made for local adaptation of newer upstream umm_malloc.
In umm_malloc.c
Renamed to umm_malloc.cpp
Added `extern "C" { ... };` around code.
Surround DBGLOG_LEVEL with #ifndef... Define value of DBGLOG_LEVEL from
umm_malloc_cfg.h
umm_realloc() - Added UMM_CRITICAL_SUSPEND()/UMM_CRITICAL_RESUME() for when
lightweight locks are available. eg. sti/cli. Single threaded single CPU
case.
umm_realloc() - appears to have been refactored to minimize memmove and
memcpy. The old version would always combine an adjacent block in the
direction of the start of the heap when available and do a memmove. This
had a defragging effect. This appears to have been replaced with an attempt
to minimize copy when possible.
Added heap stats tracking.
In umm_info.c
umm_info() - Added UMM_CRITICAL_DECL(id_info), updated critical sections
with tag.
Carried forward: Added NULL ptr check at beginning (umm_malloc.c).
In umm_poison.c:
Resolved C++ compiler error reported on get_poisoned(), and get_unpoisoned().
They now take in void * arg instead of unsigned char *.
Added #if ... || defined(UMM_POISON_CHECK_LITE) to the conditional.
In umm_integrity.c:
Replaced printf with DBGLOG_FUNCTION. This needs to be a malloc free
function and ISR safe.
Added critical sections.
In umm_malloc_cfg.h:
Added macro UMM_CRITICAL_SUSPEND()/UMM_CRITICAL_RESUME()
Globally change across all files %i to %d: umm_info.c, umm_malloc.c,
Added a #ifdef BUILD_UMM_MALLOC_C fence to prevent Arduino IDE from building
the various .c files that are #included into umm_malloc.cpp. They are
normally enabled by #define <feature name> in umm_malloc_cfg.h. In this
case it builds fine; however, if the define is global, the IDE will try and
build the .c by itself.
Notes,
umm_integrity_check() is called by macro INTEGRITY_CHECK which returns 1
on success. No corruption. Does a time consuming scan of the whole heap.
It will call UMM_HEAP_CORRUPTION_CB if an error is found.
umm_poison_check(), formerly known as check_poison_all_blocks(),
is called by macro POISON_CHECK which returns 1 on success for no
corruption. Does a time consuming scan of all active allocations for
modified poison. The new upstream version does *NOT* call
UMM_HEAP_CORRUPTION_CB if an error is found. The option description says
it does!
umm_poison_realloc() and umm_poison_free() no longer call the macro
UMM_HEAP_CORRUPTION_CB on poison error. Just a printf message is
generated. I have added alternative functions umm_poison_free_fl,
umm_poison_realloc_fl, and get_unpoisoned_check_neighbors in
umm_local.cpp. These expand the poison check on the current allocation to
include its nearest allocated neighbors in the heap.
umm_malloc() has been extended to call check_poison_neighbors for the
allocation it selects, conditionally for UMM_POISON_CHECK_LITE.
For upstream umm_malloc "# define POISON_CHECK() 0" should have been 1
add to list to report.
==============================================================================
Notes from searching for the best print option
Printing from the malloc routines is tricky. Since a print library
might call *alloc. Then recursion may follow as each error call may fail
into another error and so on.
Objective: To be able to print "last gasp" diagnostic messages
when interrupts are disabled and w/o availability of heap resources.
It turns out things are more complicated than that. These are three cases for
printing from the heap and the current solution.:
1. Printing detailed heap info through `umm_info(NULL, 1);`. This function
resides in flash and can only be called from non-ISR context. It can use
PROGMEM strings. Because SPI bus will not be busy when called from foreground.
At this time it is believed that, while running from foreground, a cache-miss
at INTLEVEL 15 can be handled. The key factor being the SPI bus is not
busy at the time of the cache-miss. It is not clear what gets invoked to
process the cache-miss. A software vector call? A hardware assisted transfer?
In any case `umm_info_safe_printf_P()` is also in flash.
The focus here is to print w/o allocating memory and use strings
in flash to preserve DRAM.
2. Printing diagnostic messages possibly from from ISR context.
Use `ets_uart_printf()` in boot ROM.
3. Printing diagnostic messages from `heap.cpp` these printf's need to check
`system_get_os_print()` to confirm debug-output is enabled just as
`os_printf()` did.
Do test calls to `system_get_os_print()` and call `ets_uart_printf()`
in boot ROM when debug print allowed.
Considerations:
* can be called from ISR
* can be called from malloc code, cannot use malloc
* can be called from malloc code that was called from an ISR
* can be called from within a critical section, eg. xt_rsil(15);
* this may be effectively the same as being called from an ISR?
Update: Current thinking is that from foreground we have more leeway
than an ISR.
Knowns:
* ets_printf - For RTOS SDK they replaced this function with one in the SDK.
Most of the problems I can see with ets_printf center around not being
able to maintain a port to thread context. That is you cannot have one
thread using one port while another thread uses the other. In the no OS
case we cannot have one area of code using one port and another area of
code using the other port. Most of the ROM printf functions are not built
to support this kind of usage. Things get especially dangerous when you
try to use the ets_external_printf stuff.
* ets_vprintf - by itself is safe.
* newlibc printf - not safe - lives in flash.
* newlibc snprintf - not safe - lives in flash.
* builtin putc1 print function - Is installed when you use
ets_install_uart_printf. Which calls ets_install_putc1. The selection of UART
is performed by calling uart_buff_switch with 0 for UART0 and 1 for UART1.
This should work for our purpose here, if handled as follows:
* call uart_buff_switch at each printf call to reselect UART
* Update: uart_buff_switch is now updated by uart_set_debug() in uart.cpp
* use a stack buffer to hold a copy the PROGMEM string to print from.
* use ets_vprintf for printing with putc1 function.
* os_printf_plus looks interesting. It is in IRAM. If no heap is available it
will use up to 64 bytes of stack space to copy a PROGMEM fmt for printing.
Issues:
* Printing is turned off by system_set_os_print
* putc1 needs to be in IRAM - this is a uart.cpp issue
* Need to force system_get_free_heap_size to return 0 during critical periods.
* won't work for umm_info if it prints over 64 characters.
* along with umm_info there are other debug messages that exceed 64 characters.
* ets_uart_printf - Appears safe. Just no PROGMEM support. Uses
uart_buff_switch to select UART.
===============================================================================
heap.cpp is the entry point for most of the heap API calls.
It is a merge point for abstracted heap API calls, such as _malloc_r,
pvPortMalloc, and malloc. Thin wrappers are created here for these entry points
and others. The wrappers call through to equivalent umm_malloc entry-point.
These wrappers also provide the access points to do debug options, like OOM,
Integrity Check, and Poison Check.
-DEBUG_ESP_OOM or select `Debug Level: "OOM"` from the IDE.
This option will add extra code to save information on the last OOM event. If
your code is built with the `Debug port: "Serial"` option, debug messages will
print on OOM events. You do not have to do `Debug port: "Serial"` to get OOM
debug messages. From within your code, you can make a call to
`Serial.debugOutput(true);` to enable OOM printing. Of course for this to work
your code must be built with `Debug Level: "OOM"` or equal.
-DUUM_POISON is now the same as -DUMM_POISON_CHECK_LITE
This is new behavior with this updated. UMM_POISON_CHECK_LITE - checks the
allocation presented at realloc() and free(). Expands the poison check on the
current allocation to include its nearest allocated neighbors in the heap.
umm_malloc() will also check the neighbors of the selected allocation before
use.
For more details and options search on UMM_POISON_CHECK in `umm_malloc_cfg.h`
TODO: provide some interesting numbers on the time to perform:
* UMM_POISON_CHECK
* UMM_INTEGRITY_CHECK
* umm_info(NUll, 0) built with and without print capability
* umm_info(NUll, 1) printing a report to Serial device.
===============================================================================
Enhancement ideas:
1. Add tagging to heap allocations. Redefine UMM_POISONED_BLOCK_LEN_TYPE,
expand it to include an element for the calling address of allocating
requester. Expand umm_info(NULL, 1) to print the respective address with each
active allocation. The difficulty here will be the ever-growing complexity of
overlapping build options. I think it would be easiest to build this in with
and expand the UMM_POISON_CHECK_LITE option.
2. A build option to not have printing, from umm_info() compiled in. This can
save on the execution time spent with interrupts disabled.
*/
#endif

109
cores/esp8266/umm_malloc/README.md
View File

@ -10,18 +10,18 @@ might get expensive.
## Acknowledgements
Joerg Wunsch and the avr-libc provided the first malloc() implementation
Joerg Wunsch and the avr-libc provided the first `malloc()` implementation
that I examined in detail.
http://www.nongnu.org/avr-libc
`http://www.nongnu.org/avr-libc`
Doug Lea's paper on malloc() was another excellent reference and provides
a lot of detail on advanced memory management techniques such as binning.
http://g.oswego.edu/dl/html/malloc.html
`http://g.oswego.edu/dl/html/malloc.html`
Bill Dittman provided excellent suggestions, including macros to support
using these functions in critical sections, and for optimizing realloc()
using these functions in critical sections, and for optimizing `realloc()`
further by checking to see if the previous block was free and could be
used for the new block size. This can help to reduce heap fragmentation
significantly.
@ -30,11 +30,73 @@ Yaniv Ankin suggested that a way to dump the current heap condition
might be useful. I combined this with an idea from plarroy to also
allow checking a free pointer to make sure it's valid.
Dimitry Frank contributed many helpful additions to make things more
robust including a user specified config file and a method of testing
the integrity of the data structures.
## Usage
Copy the `umm_malloc_cfg_example.h` file to `umm_malloc_cfg.h` and
make the changes required to support your application.
The following `#define`s must be set to something useful for the
library to work at all
- `UMM_MALLOC_CFG_HEAP_ADDR` must be set to the symbol representing
the starting address of the heap. The heap must be
aligned on the natural boundary size of the processor.
- `UMM_MALLOC_CFG_HEAP_SIZE` must be set to the size of the heap.
The heap size must be a multiple of the natural boundary size of
the processor.
The fit algorithm is defined as either:
- `UMM_BEST_FIT` which scans the entire free list and looks
for either an exact fit or the smallest block that will
satisfy the request. This is the default fit method.
- `UMM_FIRST_FIT` which scans the entire free list and looks
for the first block that satisfies the request.
The following `#define`s are disabled by default and should
remain disabled for production use. They are helpful when
testing allocation errors (which are normally due to bugs in
the application code) or for running the test suite when
making changes to the code.
You can define them in your compiler command line or uncomment
the corresponding entries is `umm_malloc_cfg.h`:
- `UMM_INFO` is used to include code that allows dumping
the entire heap structure (helpful when there's a problem).
- `UMM_INTEGRITY_CHECK` is used to include code that
performs an integrity check on the heap structure. It's
up to you to call the `umm_integrity_check()` function.
- `UMM_POISON_CHECK` is used to include code that
adds some bytes around the memory being allocated that
are filled with known data. If the data is not intact
when the block is checked, then somone has written outside
of the memory block they have been allocated. It is up
to you to call the `umm_poison_check()` function.
## API
The following functions are available for your application:
- `void *umm_malloc( size_t size );`
- `void *umm_calloc( size_t num, size_t size );`
- `void *umm_realloc( void *ptr, size_t size );`
- `void umm_free( void *ptr );`
They have exactly the same semantics as the corresponding standard library
functions.
## Background
The memory manager assumes the following things:
1. The standard POSIX compliant malloc/realloc/free semantics are used
1. The standard POSIX compliant malloc/calloc/realloc/free semantics are used
1. All memory used by the manager is allocated at link time, it is aligned
on a 32 bit boundary, it is contiguous, and its extent (start and end
address) is filled in by the linker.
@ -45,17 +107,17 @@ The fastest linked list implementations use doubly linked lists so that
its possible to insert and delete blocks in constant time. This memory
manager keeps track of both free and used blocks in a doubly linked list.
Most memory managers use some kind of list structure made up of pointers
Most memory managers use a list structure made up of pointers
to keep track of used - and sometimes free - blocks of memory. In an
embedded system, this can get pretty expensive as each pointer can use
up to 32 bits.
In most embedded systems there is no need for managing large blocks
of memory dynamically, so a full 32 bit pointer based data structure
In most embedded systems there is no need for managing a large quantity
of memory blocks dynamically, so a full 32 bit pointer based data structure
for the free and used block lists is wasteful. A block of memory on
the free list would use 16 bytes just for the pointers!
This memory management library sees the malloc heap as an array of blocks,
This memory management library sees the heap as an array of blocks,
and uses block numbers to keep track of locations. The block numbers are
15 bits - which allows for up to 32767 blocks of memory. The high order
bit marks a block as being either free or in use, which will be explained
@ -75,7 +137,7 @@ can always add more data bytes to the body of the memory block
at the expense of free block size overhead.
There are a lot of little features and optimizations in this memory
management system that makes it especially suited to small embedded, but
management system that makes it especially suited to small systems, and
the best way to appreciate them is to review the data structures and
algorithms used, so let's get started.
@ -124,10 +186,9 @@ Where:
- n is the index of the next block in the heap
- p is the index of the previous block in the heap
Note that the free list information is gone, because it's now being used to
store actual data for the application. It would have been nice to store
the next and previous free list indexes as well, but that would be a waste
of space. If we had even 500 items in use, that would be 2,000 bytes for
Note that the free list information is gone because it's now
being used to store actual data for the application. If we had
even 500 items in use, that would be 2,000 bytes for
free list information. We simply can't afford to waste that much.
The address of the `...` area is what is returned to the application
@ -143,7 +204,7 @@ described.
`...` between memory blocks indicates zero or more additional blocks are
allocated for use by the upper block.
And while we're talking about "upper" and "lower" blocks, we should make
While we're talking about "upper" and "lower" blocks, we should make
a comment about adresses. In the diagrams, a block higher up in the
picture is at a lower address. And the blocks grow downwards their
block index increases as does their physical address.
@ -166,24 +227,27 @@ we're at the end of the list!
At this point, the malloc has a special test that checks if the current
block index is 0, which it is. This special case initializes the free
list to point at block index 1.
list to point at block index 1 and then points block 1 to the
last block (lf) on the heap.
```
BEFORE AFTER
+----+----+----+----+ +----+----+----+----+
0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 |
+----+----+----+----+ +----+----+----+----+
+----+----+----+----+
1 | 0 | 0 | 0 | 0 |
1 |*lf | 0 | 0 | 0 |
+----+----+----+----+
...
+----+----+----+----+
lf | 0 | 1 | 0 | 0 |
+----+----+----+----+
```
The heap is now ready to complete the first malloc operation.
### Operation of malloc when we have reached the end of the free list and
there is no block large enough to accommodate the request.
### Operation of malloc when we have reached the end of the free list and there is no block large enough to accommodate the request.
This happens at the very first malloc operation, or any time the free
list is traversed and no free block large enough for the request is
@ -306,8 +370,7 @@ else
```
Step 1 of the free operation checks if the next block is free, and if it
is then insert this block into the free list and assimilate the next block
with this one.
is assimilate the next block with this one.
Note that c is the block we are freeing up, cf is the free block that
follows it.

21
cores/esp8266/umm_malloc/dbglog/LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2016 Ralph Hempel
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

1
cores/esp8266/umm_malloc/dbglog/README.txt Normal file
View File

@ -0,0 +1 @@
Downloaded from: https://github.com/rhempel/c-helper-macros/tree/develop

98
cores/esp8266/umm_malloc/dbglog/dbglog.h Normal file
View File

@ -0,0 +1,98 @@
/* ----------------------------------------------------------------------------
* dbglog.h - A set of macros that cleans up code that needs to produce debug
* or log information.
*
* Many embedded systems still put a premium on code space and therefore need
* a way to conditionally compile in debug code. Yes, it can lead to code that
* runs differently depending on whether the debug code is cmpiled in or not
* but you need to be able to evaluate the tradeoff.
*
* See copyright notice in LICENSE.TXT
* ----------------------------------------------------------------------------
* NOTE WELL that this file may be included multiple times - this allows you
* to set the trace level #define DBGLOG_LEVEL x
*
* To update which of the DBGLOG macros are compiled in, you must redefine the
* DBGLOG_LEVEL macro and the inlcude the dbglog.h file again, like this:
*
* #undef DBGLOG_LEVEL
* #define DBGLOG_LEVEL 6
* #include "dbglog/dbglog.txt"
*
* To handle multiple inclusion, we need to first undefine any macros we define
* so that the compiler does not warn us that we are changing a macro.
* ----------------------------------------------------------------------------
* The DBGLOG_LEVEL and DBGLOG_FUNCTION should be defined BEFORE this
* file is included or else the following defaults are used:
*
* #define DBGLOG_LEVEL 0
* #define DBGLOG_FUNCTION printf
* ----------------------------------------------------------------------------
* There are macros to handle the following decreasing levels of detail:
*
* 6 = TRACE
* 5 = DEBUG
* 4 = CRITICAL
* 3 = ERROR
* 2 = WARNING
* 1 = INFO
* 0 = FORCE - The DBGLOG_FUNCTION is always compiled in and is called only when
* the first parameter to the macro is non-0
* ----------------------------------------------------------------------------
*/
#undef DBGLOG_TRACE
#undef DBGLOG_DEBUG
#undef DBGLOG_CRITICAL
#undef DBGLOG_ERROR
#undef DBGLOG_WARNING
#undef DBGLOG_INFO
#undef DBGLOG_FORCE
#ifndef DBGLOG_LEVEL
# define DBGLOG_LEVEL 0
#endif
#ifndef DBGLOG_FUNCTION
# define DBGLOG_FUNCTION printf
#endif
/* ------------------------------------------------------------------------- */
#if DBGLOG_LEVEL >= 6
# define DBGLOG_TRACE(format, ...) DBGLOG_FUNCTION(format, ## __VA_ARGS__)
#else
# define DBGLOG_TRACE(format, ...)
#endif
#if DBGLOG_LEVEL >= 5
# define DBGLOG_DEBUG(format, ...) DBGLOG_FUNCTION(format, ## __VA_ARGS__)
#else
# define DBGLOG_DEBUG(format, ...)
#endif
#if DBGLOG_LEVEL >= 4
# define DBGLOG_CRITICAL(format, ...) DBGLOG_FUNCTION(format, ## __VA_ARGS__)
#else
# define DBGLOG_CRITICAL(format, ...)
#endif
#if DBGLOG_LEVEL >= 3
# define DBGLOG_ERROR(format, ...) DBGLOG_FUNCTION(format, ## __VA_ARGS__)
#else
# define DBGLOG_ERROR(format, ...)
#endif
#if DBGLOG_LEVEL >= 2
# define DBGLOG_WARNING(format, ...) DBGLOG_FUNCTION(format, ## __VA_ARGS__)
#else
# define DBGLOG_WARNING(format, ...)
#endif
#if DBGLOG_LEVEL >= 1
# define DBGLOG_INFO(format, ...) DBGLOG_FUNCTION(format, ## __VA_ARGS__)
#else
# define DBGLOG_INFO(format, ...)
#endif
#define DBGLOG_FORCE(force, format, ...) {if(force) {DBGLOG_FUNCTION(format, ## __VA_ARGS__);}}

185
cores/esp8266/umm_malloc/umm_info.c Normal file
View File

@ -0,0 +1,185 @@
#if defined(BUILD_UMM_MALLOC_C)
#ifdef UMM_INFO
/* ----------------------------------------------------------------------------
* One of the coolest things about this little library is that it's VERY
* easy to get debug information about the memory heap by simply iterating
* through all of the memory blocks.
*
* As you go through all the blocks, you can check to see if it's a free
* block by looking at the high order bit of the next block index. You can
* also see how big the block is by subtracting the next block index from
* the current block number.
*
* The umm_info function does all of that and makes the results available
* in the ummHeapInfo structure.
* ----------------------------------------------------------------------------
*/
UMM_HEAP_INFO ummHeapInfo;
void *umm_info( void *ptr, int force ) {
UMM_CRITICAL_DECL(id_info);
unsigned short int blockNo = 0;
if (umm_heap == NULL) {
umm_init();
}
/* Protect the critical section... */
UMM_CRITICAL_ENTRY(id_info);
/*
* Clear out all of the entries in the ummHeapInfo structure before doing
* any calculations..
*/
memset( &ummHeapInfo, 0, sizeof( ummHeapInfo ) );
DBGLOG_FORCE( force, "\n" );
DBGLOG_FORCE( force, "+----------+-------+--------+--------+-------+--------+--------+\n" );
DBGLOG_FORCE( force, "|0x%08lx|B %5d|NB %5d|PB %5d|Z %5d|NF %5d|PF %5d|\n",
(unsigned long)(&UMM_BLOCK(blockNo)),
blockNo,
UMM_NBLOCK(blockNo) & UMM_BLOCKNO_MASK,
UMM_PBLOCK(blockNo),
(UMM_NBLOCK(blockNo) & UMM_BLOCKNO_MASK )-blockNo,
UMM_NFREE(blockNo),
UMM_PFREE(blockNo) );
/*
* Now loop through the block lists, and keep track of the number and size
* of used and free blocks. The terminating condition is an nb pointer with
* a value of zero...
*/
blockNo = UMM_NBLOCK(blockNo) & UMM_BLOCKNO_MASK;
while( UMM_NBLOCK(blockNo) & UMM_BLOCKNO_MASK ) {
size_t curBlocks = (UMM_NBLOCK(blockNo) & UMM_BLOCKNO_MASK )-blockNo;
++ummHeapInfo.totalEntries;
ummHeapInfo.totalBlocks += curBlocks;
/* Is this a free block? */
if( UMM_NBLOCK(blockNo) & UMM_FREELIST_MASK ) {
++ummHeapInfo.freeEntries;
ummHeapInfo.freeBlocks += curBlocks;
ummHeapInfo.freeSize2 += (unsigned int)curBlocks
* (unsigned int)sizeof(umm_block)
* (unsigned int)curBlocks
* (unsigned int)sizeof(umm_block);
if (ummHeapInfo.maxFreeContiguousBlocks < curBlocks) {
ummHeapInfo.maxFreeContiguousBlocks = curBlocks;
}
DBGLOG_FORCE( force, "|0x%08lx|B %5d|NB %5d|PB %5d|Z %5u|NF %5d|PF %5d|\n",
(unsigned long)(&UMM_BLOCK(blockNo)),
blockNo,
UMM_NBLOCK(blockNo) & UMM_BLOCKNO_MASK,
UMM_PBLOCK(blockNo),
(unsigned int)curBlocks,
UMM_NFREE(blockNo),
UMM_PFREE(blockNo) );
/* Does this block address match the ptr we may be trying to free? */
if( ptr == &UMM_BLOCK(blockNo) ) {
/* Release the critical section... */
UMM_CRITICAL_EXIT(id_info);
return( ptr );
}
} else {
++ummHeapInfo.usedEntries;
ummHeapInfo.usedBlocks += curBlocks;
DBGLOG_FORCE( force, "|0x%08lx|B %5d|NB %5d|PB %5d|Z %5u|\n",
(unsigned long)(&UMM_BLOCK(blockNo)),
blockNo,
UMM_NBLOCK(blockNo) & UMM_BLOCKNO_MASK,
UMM_PBLOCK(blockNo),
(unsigned int)curBlocks );
}
blockNo = UMM_NBLOCK(blockNo) & UMM_BLOCKNO_MASK;
}
/*
* Update the accounting totals with information from the last block, the
* rest must be free!
*/
{
size_t curBlocks = UMM_NUMBLOCKS-blockNo;
ummHeapInfo.freeBlocks += curBlocks;
ummHeapInfo.totalBlocks += curBlocks;
if (ummHeapInfo.maxFreeContiguousBlocks < curBlocks) {
ummHeapInfo.maxFreeContiguousBlocks = curBlocks;
}
}
DBGLOG_FORCE( force, "|0x%08lx|B %5d|NB %5d|PB %5d|Z %5d|NF %5d|PF %5d|\n",
(unsigned long)(&UMM_BLOCK(blockNo)),
blockNo,
UMM_NBLOCK(blockNo) & UMM_BLOCKNO_MASK,
UMM_PBLOCK(blockNo),
UMM_NUMBLOCKS-blockNo,
UMM_NFREE(blockNo),
UMM_PFREE(blockNo) );
DBGLOG_FORCE( force, "+----------+-------+--------+--------+-------+--------+--------+\n" );
DBGLOG_FORCE( force, "Total Entries %5d Used Entries %5d Free Entries %5d\n",
ummHeapInfo.totalEntries,
ummHeapInfo.usedEntries,
ummHeapInfo.freeEntries );
DBGLOG_FORCE( force, "Total Blocks %5d Used Blocks %5d Free Blocks %5d\n",
ummHeapInfo.totalBlocks,
ummHeapInfo.usedBlocks,
ummHeapInfo.freeBlocks );
DBGLOG_FORCE( force, "+--------------------------------------------------------------+\n" );
#if defined(UMM_STATS) || defined(UMM_STATS_FULL)
if (ummHeapInfo.freeBlocks == ummStats.free_blocks) {
DBGLOG_FORCE( force, "heap info Free blocks and heap statistics Free blocks match.\n");
} else {
DBGLOG_FORCE( force, "\nheap info Free blocks %5d != heap statistics Free Blocks %5d\n\n",
ummHeapInfo.freeBlocks,
ummStats.free_blocks );
}
DBGLOG_FORCE( force, "+--------------------------------------------------------------+\n" );
print_stats(force);
#endif
/* Release the critical section... */
UMM_CRITICAL_EXIT(id_info);
return( NULL );
}
/* ------------------------------------------------------------------------ */
size_t umm_free_heap_size( void ) {
umm_info(NULL, 0);
return (size_t)ummHeapInfo.freeBlocks * sizeof(umm_block);
}
size_t umm_max_block_size( void ) {
umm_info(NULL, 0);
return ummHeapInfo.maxFreeContiguousBlocks * sizeof(umm_block);
}
/* ------------------------------------------------------------------------ */
#endif
#endif // defined(BUILD_UMM_MALLOC_C)

134
cores/esp8266/umm_malloc/umm_integrity.c Normal file
View File

@ -0,0 +1,134 @@
#if defined(BUILD_UMM_MALLOC_C)
/* integrity check (UMM_INTEGRITY_CHECK) {{{ */
#if defined(UMM_INTEGRITY_CHECK)
/*
* Perform integrity check of the whole heap data. Returns 1 in case of
* success, 0 otherwise.
*
* First of all, iterate through all free blocks, and check that all backlinks
* match (i.e. if block X has next free block Y, then the block Y should have
* previous free block set to X).
*
* Additionally, we check that each free block is correctly marked with
* `UMM_FREELIST_MASK` on the `next` pointer: during iteration through free
* list, we mark each free block by the same flag `UMM_FREELIST_MASK`, but
* on `prev` pointer. We'll check and unmark it later.
*
* Then, we iterate through all blocks in the heap, and similarly check that
* all backlinks match (i.e. if block X has next block Y, then the block Y
* should have previous block set to X).
*
* But before checking each backlink, we check that the `next` and `prev`
* pointers are both marked with `UMM_FREELIST_MASK`, or both unmarked.
* This way, we ensure that the free flag is in sync with the free pointers
* chain.
*/
int umm_integrity_check(void) {
UMM_CRITICAL_DECL(id_integrity);
int ok = 1;
unsigned short int prev;
unsigned short int cur;
if (umm_heap == NULL) {
umm_init();
}
/* Iterate through all free blocks */
prev = 0;
UMM_CRITICAL_ENTRY(id_integrity);
while(1) {
cur = UMM_NFREE(prev);
/* Check that next free block number is valid */
if (cur >= UMM_NUMBLOCKS) {
DBGLOG_FUNCTION("heap integrity broken: too large next free num: %d "
"(in block %d, addr 0x%lx)\n", cur, prev,
(unsigned long)&UMM_NBLOCK(prev));
ok = 0;
goto clean;
}
if (cur == 0) {
/* No more free blocks */
break;
}
/* Check if prev free block number matches */
if (UMM_PFREE(cur) != prev) {
DBGLOG_FUNCTION("heap integrity broken: free links don't match: "
"%d -> %d, but %d -> %d\n",
prev, cur, cur, UMM_PFREE(cur));
ok = 0;
goto clean;
}
UMM_PBLOCK(cur) |= UMM_FREELIST_MASK;
prev = cur;
}
/* Iterate through all blocks */
prev = 0;
while(1) {
cur = UMM_NBLOCK(prev) & UMM_BLOCKNO_MASK;
/* Check that next block number is valid */
if (cur >= UMM_NUMBLOCKS) {
DBGLOG_FUNCTION("heap integrity broken: too large next block num: %d "
"(in block %d, addr 0x%lx)\n", cur, prev,
(unsigned long)&UMM_NBLOCK(prev));
ok = 0;
goto clean;
}
if (cur == 0) {
/* No more blocks */
break;
}
/* make sure the free mark is appropriate, and unmark it */
if ((UMM_NBLOCK(cur) & UMM_FREELIST_MASK)
!= (UMM_PBLOCK(cur) & UMM_FREELIST_MASK))
{
DBGLOG_FUNCTION("heap integrity broken: mask wrong at addr 0x%lx: n=0x%x, p=0x%x\n",
(unsigned long)&UMM_NBLOCK(cur),
(UMM_NBLOCK(cur) & UMM_FREELIST_MASK),
(UMM_PBLOCK(cur) & UMM_FREELIST_MASK)
);
ok = 0;
goto clean;
}
/* make sure the block list is sequential */
if (cur <= prev ) {
DBGLOG_FUNCTION("heap integrity broken: next block %d is before prev this one "
"(in block %d, addr 0x%lx)\n", cur, prev,
(unsigned long)&UMM_NBLOCK(prev));
ok = 0;
goto clean;
}
/* unmark */
UMM_PBLOCK(cur) &= UMM_BLOCKNO_MASK;
/* Check if prev block number matches */
if (UMM_PBLOCK(cur) != prev) {
DBGLOG_FUNCTION("heap integrity broken: block links don't match: "
"%d -> %d, but %d -> %d\n",
prev, cur, cur, UMM_PBLOCK(cur));
ok = 0;
goto clean;
}
prev = cur;
}
clean:
UMM_CRITICAL_EXIT(id_integrity);
if (!ok){
UMM_HEAP_CORRUPTION_CB();
}
return ok;
}
#endif
/* }}} */
#endif // defined(BUILD_UMM_MALLOC_C)

215
cores/esp8266/umm_malloc/umm_local.c Normal file
View File

@ -0,0 +1,215 @@
/*
* Local Additions/Enhancements
*
*/
#if defined(BUILD_UMM_MALLOC_C)
#if defined(UMM_CRITICAL_METRICS)
/*
* umm_malloc performance measurments for critical sections
*/
UMM_TIME_STATS time_stats = {
{0xFFFFFFFF, 0U, 0U, 0U},
{0xFFFFFFFF, 0U, 0U, 0U},
{0xFFFFFFFF, 0U, 0U, 0U},
#ifdef UMM_INFO
{0xFFFFFFFF, 0U, 0U, 0U},
#endif
#ifdef UMM_POISON_CHECK
{0xFFFFFFFF, 0U, 0U, 0U},
#endif
#ifdef UMM_INTEGRITY_CHECK
{0xFFFFFFFF, 0U, 0U, 0U},
#endif
{0xFFFFFFFF, 0U, 0U, 0U} };
bool ICACHE_FLASH_ATTR get_umm_get_perf_data(UMM_TIME_STATS *p, size_t size)
{
UMM_CRITICAL_DECL(id_no_tag);
if (p && sizeof(time_stats) == size)
{
UMM_CRITICAL_ENTRY(id_no_tag);
memcpy(p, &time_stats, size);
UMM_CRITICAL_EXIT(id_no_tag);
return true;
}
return false;
}
#endif
// Alternate Poison functions
#if defined(UMM_POISON_CHECK_LITE)
// We skip this when doing the full poison check.
static int check_poison_neighbors( unsigned short cur ) {
unsigned short int c;
if ( 0 == cur )
return 1;
c = UMM_PBLOCK(cur) & UMM_BLOCKNO_MASK;
while( c && (UMM_NBLOCK(c) & UMM_BLOCKNO_MASK) ) {
/*
There can be up to 1 free block neighbor in either direction.
This loop should self limit to 2 passes, due to heap design.
i.e. Adjacent free space is always consolidated.
*/
if ( !(UMM_NBLOCK(c) & UMM_FREELIST_MASK) ) {
if ( !check_poison_block(&UMM_BLOCK(c)) )
return 0;
break;
}
c = UMM_PBLOCK(c) & UMM_BLOCKNO_MASK;
}
c = UMM_NBLOCK(cur) & UMM_BLOCKNO_MASK;
while( (UMM_NBLOCK(c) & UMM_BLOCKNO_MASK) ) {
if ( !(UMM_NBLOCK(c) & UMM_FREELIST_MASK) ) {
if ( !check_poison_block(&UMM_BLOCK(c)) )
return 0;
break;
}
c = UMM_NBLOCK(c) & UMM_BLOCKNO_MASK;
}
return 1;
}
#endif
#if defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE)
/* ------------------------------------------------------------------------ */
static void *get_unpoisoned_check_neighbors( void *v_ptr, const char* file, int line ) {
unsigned char *ptr = (unsigned char *)v_ptr;
if (ptr != NULL) {
ptr -= (sizeof(UMM_POISONED_BLOCK_LEN_TYPE) + UMM_POISON_SIZE_BEFORE);
#if defined(UMM_POISON_CHECK_LITE)
UMM_CRITICAL_DECL(id_poison);
unsigned short int c;
bool poison = false;
/* Figure out which block we're in. Note the use of truncated division... */
c = (((char *)ptr)-(char *)(&(umm_heap[0])))/sizeof(umm_block);
UMM_CRITICAL_ENTRY(id_poison);
poison = check_poison_block(&UMM_BLOCK(c)) && check_poison_neighbors(c);
UMM_CRITICAL_EXIT(id_poison);
if (!poison) {
if (file) {
__panic_func(file, line, "");
} else {
abort();
}
}
#else
/*
* No need to check poison here. POISON_CHECK() has already done a
* full heap check.
*/
(void)file;
(void)line;
#endif
}
return (void *)ptr;
}
/* ------------------------------------------------------------------------ */
void *umm_poison_realloc_fl(void *ptr, size_t size, const char* file, int line) {
void *ret;
ptr = get_unpoisoned_check_neighbors(ptr, file, line);
size += poison_size(size);
ret = umm_realloc(ptr, size);
ret = get_poisoned(ret, size);
return ret;
}
/* ------------------------------------------------------------------------ */
void umm_poison_free_fl(void *ptr, const char* file, int line) {
ptr = get_unpoisoned_check_neighbors(ptr, file, line);
umm_free(ptr);
}
#endif
/* ------------------------------------------------------------------------ */
#if defined(UMM_STATS) || defined(UMM_STATS_FULL) || defined(UMM_INFO)
size_t umm_block_size( void ) {
return sizeof(umm_block);
}
#endif
#if defined(UMM_STATS) || defined(UMM_STATS_FULL)
UMM_STATISTICS ummStats;
// Keep complete call path in IRAM
size_t umm_free_heap_size_lw( void ) {
return (size_t)ummStats.free_blocks * sizeof(umm_block);
}
#endif
/*
I assume xPortGetFreeHeapSize needs to be in IRAM. Since
system_get_free_heap_size is in IRAM. Which would mean, umm_free_heap_size()
in flash, was not a safe alternative for returning the same information.
*/
#if defined(UMM_STATS) || defined(UMM_STATS_FULL)
size_t xPortGetFreeHeapSize(void) __attribute__ ((alias("umm_free_heap_size_lw")));
#elif defined(UMM_INFO)
#warning "No ISR safe function available to implement xPortGetFreeHeapSize()"
size_t xPortGetFreeHeapSize(void) __attribute__ ((alias("umm_free_heap_size")));
#endif
#if defined(UMM_STATS) || defined(UMM_STATS_FULL)
void print_stats(int force) {
DBGLOG_FORCE( force, "umm heap statistics:\n");
DBGLOG_FORCE( force, " Free Space %5u\n", ummStats.free_blocks * sizeof(umm_block));
DBGLOG_FORCE( force, " OOM Count %5u\n", ummStats.oom_count);
#if defined(UMM_STATS_FULL)
DBGLOG_FORCE( force, " Low Watermark %5u\n", ummStats.free_blocks_min * sizeof(umm_block));
DBGLOG_FORCE( force, " Low Watermark ISR %5u\n", ummStats.free_blocks_isr_min * sizeof(umm_block));
DBGLOG_FORCE( force, " MAX Alloc Request %5u\n", ummStats.alloc_max_size);
#endif
DBGLOG_FORCE( force, " Size of umm_block %5u\n", sizeof(umm_block));
DBGLOG_FORCE( force, "+--------------------------------------------------------------+\n" );
}
#endif
int ICACHE_FLASH_ATTR umm_info_safe_printf_P(const char *fmt, ...) {
/*
To use ets_strlen() and ets_strcpy() safely with PROGMEM, flash storage,
the PROGMEM address must be word (4 bytes) aligned. The destination
address for ets_memcpy must also be word-aligned.
*/
char ram_buf[ets_strlen(fmt)] __attribute__ ((aligned(4)));
ets_strcpy(ram_buf, fmt);
va_list argPtr;
va_start(argPtr, fmt);
int result = ets_vprintf(ets_uart_putc1, ram_buf, argPtr);
va_end(argPtr);
return result;
}
#endif // BUILD_UMM_MALLOC_C

54
cores/esp8266/umm_malloc/umm_local.h Normal file
View File

@ -0,0 +1,54 @@
#ifndef _UMM_LOCAL_H
#define _UMM_LOCAL_H
/*
* A home for local items exclusive to umm_malloc.c and not to be shared in
* umm_malloc_cfg.h. And, not for upstream version.
* Also used to redefine defines made in upstream files we donet want to edit.
*
*/
#undef memcpy
#undef memmove
#undef memset
#define memcpy ets_memcpy
#define memmove ets_memmove
#define memset ets_memset
/*
* This redefines DBGLOG_FORCE defined in dbglog/dbglog.h
* Just for printing from umm_info() which is assumed to always be called from
* non-ISR. Thus SPI bus is available to handle cache-miss and reading a flash
* string while INTLEVEL is non-zero.
*/
#undef DBGLOG_FORCE
#define DBGLOG_FORCE(force, format, ...) {if(force) {UMM_INFO_PRINTF(format, ## __VA_ARGS__);}}
// #define DBGLOG_FORCE(force, format, ...) {if(force) {::printf(PSTR(format), ## __VA_ARGS__);}}
#if defined(DEBUG_ESP_OOM) || defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE) || defined(UMM_INTEGRITY_CHECK)
#else
#define umm_malloc(s) malloc(s)
#define umm_calloc(n,s) calloc(n,s)
#define umm_realloc(p,s) realloc(p,s)
#define umm_free(p) free(p)
#endif
#if defined(UMM_POISON_CHECK_LITE)
static int check_poison_neighbors( unsigned short cur );
#endif
#if defined(UMM_STATS) || defined(UMM_STATS_FULL)
void ICACHE_FLASH_ATTR print_stats(int force);
#endif
int ICACHE_FLASH_ATTR umm_info_safe_printf_P(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
#define UMM_INFO_PRINTF(fmt, ...) umm_info_safe_printf_P(PSTR(fmt), ##__VA_ARGS__)
#endif

1912
cores/esp8266/umm_malloc/umm_malloc.cpp
View File

File diff suppressed because it is too large Load Diff

29
cores/esp8266/umm_malloc/umm_malloc.h
View File

@ -10,41 +10,18 @@
/* ------------------------------------------------------------------------ */
//C This include is not in upstream neither are the #ifdef __cplusplus
#include "umm_malloc_cfg.h" /* user-dependent */
#ifdef __cplusplus
extern "C" {
#endif
typedef struct UMM_HEAP_INFO_t {
unsigned short int totalEntries;
unsigned short int usedEntries;
unsigned short int freeEntries;
unsigned short int totalBlocks;
unsigned short int usedBlocks;
unsigned short int freeBlocks;
unsigned short int maxFreeContiguousBlocks;
unsigned int freeSize2;
}
UMM_HEAP_INFO;
extern UMM_HEAP_INFO ummHeapInfo;
void umm_init( void );
void *umm_info( void *ptr, int force );
void umm_init( void );
void *umm_malloc( size_t size );
void *umm_calloc( size_t num, size_t size );
void *umm_realloc( void *ptr, size_t size );
void umm_free( void *ptr );
size_t umm_free_heap_size( void );
size_t umm_max_block_size( void );
size_t umm_block_size( void );
void umm_free( void *ptr );
#ifdef __cplusplus
}

633
cores/esp8266/umm_malloc/umm_malloc_cfg.h
View File

@ -1,11 +1,17 @@
/*
* Configuration for umm_malloc
* Configuration for umm_malloc - target Arduino ESP8266 core
*
* Changes specific to a target platform go here.
*
*/
#ifndef _UMM_MALLOC_CFG_H
#define _UMM_MALLOC_CFG_H
#include <debug.h>
#include <pgmspace.h>
#include <esp8266_undocumented.h>
#ifdef __cplusplus
extern "C" {
#endif
@ -15,19 +21,6 @@ extern "C" {
#include <osapi.h>
#include "c_types.h"
#include "umm_performance.h"
#include "umm_stats.h"
#undef DBGLOG_FUNCTION
#if defined(DEBUG_ESP_PORT) || defined(DEBUG_ESP_ISR)
int _isr_safe_printf_P(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
// Note, _isr_safe_printf_P will not handle additional string arguments in
// PROGMEM. Only the 1st parameter, fmt, is supported in PROGMEM.
#define DBGLOG_FUNCTION(fmt, ...) _isr_safe_printf_P(PSTR(fmt), ##__VA_ARGS__)
#else
// Macro to place constant strings into PROGMEM and print them properly
#define DBGLOG_FUNCTION(fmt, ...) printf(PSTR(fmt), ## __VA_ARGS__ )
#endif
/*
* There are a number of defines you can set at compile time that affect how
@ -37,19 +30,7 @@ int _isr_safe_printf_P(const char *fmt, ...) __attribute__((format(printf, 1, 2)
*
* -D UMM_TEST_MAIN
*
* Set this if you want to compile in the test suite at the end of this file.
*
* If you leave this define unset, then you might want to set another one:
*
* -D UMM_REDEFINE_MEM_FUNCTIONS
*
* If you leave this define unset, then the function names are left alone as
* umm_malloc() umm_free() and umm_realloc() so that they cannot be confused
* with the C runtime functions malloc() free() and realloc()
*
* If you do set this define, then the function names become malloc()
* free() and realloc() so that they can be used as the C runtime functions
* in an embedded environment.
* Set this if you want to compile in the test suite
*
* -D UMM_BEST_FIT (defualt)
*
@ -75,46 +56,300 @@ int _isr_safe_printf_P(const char *fmt, ...) __attribute__((format(printf, 1, 2)
* ----------------------------------------------------------------------------
*/
/////////////////////////////////////////////////
#ifdef DEBUG_ESP_OOM
#define MEMLEAK_DEBUG
// umm_*alloc are not renamed to *alloc
void *umm_malloc( size_t size );
void *umm_calloc( size_t num, size_t size );
void *umm_realloc( void *ptr, size_t size );
#define umm_free free
#define umm_zalloc(s) umm_calloc(1,s)
void* malloc_loc (size_t s, const char* file, int line);
void* calloc_loc (size_t n, size_t s, const char* file, int line);
void* realloc_loc (void* p, size_t s, const char* file, int line);
// *alloc are macro calling *alloc_loc calling+checking umm_*alloc()
// they are defined at the bottom of this file
/////////////////////////////////////////////////
#else // !defined(ESP_DEBUG_OOM)
// umm_*alloc are renamed to *alloc
#define UMM_REDEFINE_MEM_FUNCTIONS
#ifdef TEST_BUILD
extern char test_umm_heap[];
#endif
#define UMM_BEST_FIT
#ifdef TEST_BUILD
/* Start addresses and the size of the heap */
#define UMM_MALLOC_CFG_HEAP_ADDR (test_umm_heap)
#define UMM_MALLOC_CFG_HEAP_SIZE 0x10000
#else
/* Start addresses and the size of the heap */
extern char _heap_start[];
#define UMM_MALLOC_CFG__HEAP_ADDR ((uint32_t)&_heap_start)
#define UMM_MALLOC_CFG__HEAP_SIZE ((size_t)(0x3fffc000 - UMM_MALLOC_CFG__HEAP_ADDR))
#define UMM_MALLOC_CFG_HEAP_ADDR ((uint32_t)&_heap_start[0])
#define UMM_MALLOC_CFG_HEAP_SIZE ((size_t)(0x3fffc000 - UMM_MALLOC_CFG_HEAP_ADDR))
#endif
/* A couple of macros to make packing structures less compiler dependent */
#define UMM_H_ATTPACKPRE
#define UMM_H_ATTPACKSUF __attribute__((__packed__))
#define UMM_BEST_FIT
#undef UMM_FIRST_FIT
/*
* -D UMM_INFO :
*
* Enables a dup of the heap contents and a function to return the total
* heap size that is unallocated - note this is not the same as the largest
* unallocated block on the heap!
*/
#define UMM_INFO
#ifdef UMM_INFO
typedef struct UMM_HEAP_INFO_t {
unsigned short int totalEntries;
unsigned short int usedEntries;
unsigned short int freeEntries;
unsigned short int totalBlocks;
unsigned short int usedBlocks;
unsigned short int freeBlocks;
unsigned short int maxFreeContiguousBlocks;
unsigned int freeSize2;
}
UMM_HEAP_INFO;
extern UMM_HEAP_INFO ummHeapInfo;
void ICACHE_FLASH_ATTR *umm_info( void *ptr, int force );
size_t ICACHE_FLASH_ATTR umm_free_heap_size( void );
size_t ICACHE_FLASH_ATTR umm_max_block_size( void );
#else
#endif
/*
* -D UMM_STATS :
* -D UMM_STATS_FULL
*
* This option provides a lightweight alternative to using `umm_info` just for
* getting `umm_free_heap_size`. With this option, a "free blocks" value is
* updated on each call to malloc/free/realloc. This option does not offer all
* the information that `umm_info` would have generated.
*
* This option is good for cases where the free heap is checked frequently. An
* example is when an app closely monitors free heap to detect memory leaks. In
* this case a single-core CPUs interrupt processing would have suffered the
* most.
*
* UMM_STATS_FULL provides additional heap statistics. It can be used to gain
* additional insight into heap usage. This option would add an additional 132
* bytes of IRAM.
*
* Status: TODO: Needs to be proposed for upstream.
*/
/*
#define UMM_STATS
#define UMM_STATS_FULL
*/
/*
* For the ESP8266 we want at lest UMM_STATS built, so we have an ISR safe
* function to call for implementing xPortGetFreeHeapSize(), because umm_info()
* is in flash.
*/
#if !defined(UMM_STATS) && !defined(UMM_STATS_FULL)
#define UMM_STATS
#endif
#if defined(UMM_STATS) && defined(UMM_STATS_FULL)
#undef UMM_STATS
#endif
#if defined(UMM_STATS) || defined(UMM_STATS_FULL)
typedef struct UMM_STATISTICS_t {
unsigned short int free_blocks;
size_t oom_count;
#ifdef UMM_STATS_FULL
unsigned short int free_blocks_min;
unsigned short int free_blocks_isr_min;
size_t alloc_max_size;
size_t last_alloc_size;
size_t id_malloc_count;
size_t id_malloc_zero_count;
size_t id_realloc_count;
size_t id_realloc_zero_count;
size_t id_free_count;
size_t id_free_null_count;
#endif
}
UMM_STATISTICS;
extern UMM_STATISTICS ummStats;
#define STATS__FREE_BLOCKS_UPDATE(s) ummStats.free_blocks += (s)
#define STATS__OOM_UPDATE() ummStats.oom_count += 1
size_t umm_free_heap_size_lw( void );
static inline size_t ICACHE_FLASH_ATTR umm_get_oom_count( void ) {
return ummStats.oom_count;
}
#else // not UMM_STATS or UMM_STATS_FULL
#define STATS__FREE_BLOCKS_UPDATE(s) (void)(s)
#define STATS__OOM_UPDATE() (void)0
#endif
#if defined(UMM_STATS) || defined(UMM_STATS_FULL) || defined(UMM_INFO)
size_t ICACHE_FLASH_ATTR umm_block_size( void );
#endif
#ifdef UMM_STATS_FULL
#define STATS__FREE_BLOCKS_MIN() \
do { \
if (ummStats.free_blocks < ummStats.free_blocks_min) \
ummStats.free_blocks_min = ummStats.free_blocks; \
} while(false)
#define STATS__FREE_BLOCKS_ISR_MIN() \
do { \
if (ummStats.free_blocks < ummStats.free_blocks_isr_min) \
ummStats.free_blocks_isr_min = ummStats.free_blocks; \
} while(false)
#define STATS__ALLOC_REQUEST(tag, s) \
do { \
ummStats.tag##_count += 1; \
ummStats.last_alloc_size = s; \
if (ummStats.alloc_max_size < s) \
ummStats.alloc_max_size = s; \
} while(false)
#define STATS__ZERO_ALLOC_REQUEST(tag, s) \
do { \
ummStats.tag##_zero_count += 1; \
} while(false)
#define STATS__NULL_FREE_REQUEST(tag) \
do { \
ummStats.tag##_null_count += 1; \
} while(false)
#define STATS__FREE_REQUEST(tag) \
do { \
ummStats.tag##_count += 1; \
} while(false)
static inline size_t ICACHE_FLASH_ATTR umm_free_heap_size_lw_min( void ) {
return (size_t)ummStats.free_blocks_min * umm_block_size();
}
static inline size_t ICACHE_FLASH_ATTR umm_free_heap_size_min_reset( void ) {
ummStats.free_blocks_min = ummStats.free_blocks;
return (size_t)ummStats.free_blocks_min * umm_block_size();
}
static inline size_t ICACHE_FLASH_ATTR umm_free_heap_size_min( void ) {
return ummStats.free_blocks_min * umm_block_size();
}
static inline size_t ICACHE_FLASH_ATTR umm_free_heap_size_isr_min( void ) {
return ummStats.free_blocks_isr_min * umm_block_size();
}
static inline size_t ICACHE_FLASH_ATTR umm_get_max_alloc_size( void ) {
return ummStats.alloc_max_size;
}
static inline size_t ICACHE_FLASH_ATTR umm_get_last_alloc_size( void ) {
return ummStats.last_alloc_size;
}
static inline size_t ICACHE_FLASH_ATTR umm_get_malloc_count( void ) {
return ummStats.id_malloc_count;
}
static inline size_t ICACHE_FLASH_ATTR umm_get_malloc_zero_count( void ) {
return ummStats.id_malloc_zero_count;
}
static inline size_t ICACHE_FLASH_ATTR umm_get_realloc_count( void ) {
return ummStats.id_realloc_count;
}
static inline size_t ICACHE_FLASH_ATTR umm_get_realloc_zero_count( void ) {
return ummStats.id_realloc_zero_count;
}
static inline size_t ICACHE_FLASH_ATTR umm_get_free_count( void ) {
return ummStats.id_free_count;
}
static inline size_t ICACHE_FLASH_ATTR umm_get_free_null_count( void ) {
return ummStats.id_free_null_count;
}
#else // Not UMM_STATS_FULL
#define STATS__FREE_BLOCKS_MIN() (void)0
#define STATS__FREE_BLOCKS_ISR_MIN() (void)0
#define STATS__ALLOC_REQUEST(tag, s) (void)(s)
#define STATS__ZERO_ALLOC_REQUEST(tag, s) (void)(s)
#define STATS__NULL_FREE_REQUEST(tag) (void)0
#define STATS__FREE_REQUEST(tag) (void)0
#endif
/*
Per Devyte, the core currently doesn't support masking a specific interrupt
level. That doesn't mean it can't be implemented, only that at this time
locking is implemented as all or nothing.
https://github.com/esp8266/Arduino/issues/6246#issuecomment-508612609
So for now we default to all, 15.
*/
#ifndef DEFAULT_CRITICAL_SECTION_INTLEVEL
#define DEFAULT_CRITICAL_SECTION_INTLEVEL 15
#endif
/*
* -D UMM_CRITICAL_METRICS
*
* Build option to collect timing usage data on critical section usage in
* functions: info, malloc, realloc. Collects MIN, MAX, and number of time IRQs
* were disabled at request time. Note, for realloc MAX disabled time will
* include the time spent in calling malloc and/or free. Examine code for
* specifics on what info is available and how to access.
*
* Status: TODO: Needs to be proposed for upstream. Also should include updates
* to UMM_POISON_CHECK and UMM_INTEGRITY_CHECK to include a critical section.
*/
/*
#define UMM_CRITICAL_METRICS
*/
#if defined(UMM_CRITICAL_METRICS)
// This option adds support for gathering time locked data
typedef struct UMM_TIME_STAT_t {
uint32_t min;
uint32_t max;
uint32_t start;
uint32_t intlevel;
}
UMM_TIME_STAT;
typedef struct UMM_TIME_STATS_t UMM_TIME_STATS;
extern UMM_TIME_STATS time_stats;
bool get_umm_get_perf_data(UMM_TIME_STATS *p, size_t size);
static inline void _critical_entry(UMM_TIME_STAT *p, uint32_t *saved_ps) {
*saved_ps = xt_rsil(DEFAULT_CRITICAL_SECTION_INTLEVEL);
if (0U != (*saved_ps & 0x0FU)) {
p->intlevel += 1U;
}
p->start = esp_get_cycle_count();
}
static inline void _critical_exit(UMM_TIME_STAT *p, uint32_t *saved_ps) {
uint32_t elapse = esp_get_cycle_count() - p->start;
if (elapse < p->min)
p->min = elapse;
if (elapse > p->max)
p->max = elapse;
xt_wsr_ps(*saved_ps);
}
#endif
/*
* A couple of macros to make it easier to protect the memory allocator
* in a multitasking system. You should set these macros up to use whatever
@ -125,23 +360,82 @@ extern char _heap_start[];
* called from within umm_malloc()
*/
#if defined(UMM_CRITICAL_PERIOD_ANALYZE)
#define UMM_CRITICAL_DECL(tag) uint32_t _saved_ps_##tag
#define UMM_CRITICAL_ENTRY(tag) _critical_entry(&time_stats.tag, &_saved_ps_##tag)
#define UMM_CRITICAL_EXIT(tag) _critical_exit(&time_stats.tag, &_saved_ps_##tag)
#ifdef TEST_BUILD
extern int umm_critical_depth;
extern int umm_max_critical_depth;
#define UMM_CRITICAL_ENTRY() {\
++umm_critical_depth; \
if (umm_critical_depth > umm_max_critical_depth) { \
umm_max_critical_depth = umm_critical_depth; \
} \
}
#define UMM_CRITICAL_EXIT() (umm_critical_depth--)
#else
#if defined(UMM_CRITICAL_METRICS)
#define UMM_CRITICAL_DECL(tag) uint32_t _saved_ps_##tag
#define UMM_CRITICAL_ENTRY(tag)_critical_entry(&time_stats.tag, &_saved_ps_##tag)
#define UMM_CRITICAL_EXIT(tag) _critical_exit(&time_stats.tag, &_saved_ps_##tag)
// This method preserves the intlevel on entry and restores the
// original intlevel at exit.
#define UMM_CRITICAL_DECL(tag) uint32_t _saved_ps_##tag
#define UMM_CRITICAL_ENTRY(tag) _saved_ps_##tag = xt_rsil(DEFAULT_CRITICAL_SECTION_INTLEVEL)
#define UMM_CRITICAL_EXIT(tag) xt_wsr_ps(_saved_ps_##tag)
#else // ! UMM_CRITICAL_METRICS
// This method preserves the intlevel on entry and restores the
// original intlevel at exit.
#define UMM_CRITICAL_DECL(tag) uint32_t _saved_ps_##tag
#define UMM_CRITICAL_ENTRY(tag) _saved_ps_##tag = xt_rsil(DEFAULT_CRITICAL_SECTION_INTLEVEL)
#define UMM_CRITICAL_EXIT(tag) xt_wsr_ps(_saved_ps_##tag)
#endif
#endif
/*
* -D UMM_LIGHTWEIGHT_CPU
*
* The use of this macro is hardware/application specific.
*
* With some CPUs, the only available method for locking are the instructions
* for interrupts disable/enable. These macros are meant for lightweight single
* CPU systems that are sensitive to interrupts being turned off for too long. A
* typically UMM_CRITICAL_ENTRY would save current IRQ state then disable IRQs.
* Then UMM_CRITICAL_EXIT would restore previous IRQ state. This option adds
* additional critical entry/exit points by the method of defining the macros
* UMM_CRITICAL_SUSPEND and UMM_CRITICAL_RESUME to the values of
* UMM_CRITICAL_EXIT and UMM_CRITICAL_ENTRY. These additional exit/entries
* allow time to service interrupts during the reentrant sections of the code.
*
* Performance may be impacked if used with multicore CPUs. The higher frquency
* of locking and unlocking may be an issue with locking methods that have a
* high overhead.
*
* Status: TODO: Needs to be proposed for upstream.
*/
/*
*/
#define UMM_LIGHTWEIGHT_CPU
#ifdef UMM_LIGHTWEIGHT_CPU
#define UMM_CRITICAL_SUSPEND(tag) UMM_CRITICAL_EXIT(tag)
#define UMM_CRITICAL_RESUME(tag) UMM_CRITICAL_ENTRY(tag)
#else
#define UMM_CRITICAL_SUSPEND(tag) do {} while(0)
#define UMM_CRITICAL_RESUME(tag) do {} while(0)
#endif
/*
* -D UMM_REALLOC_MINIMIZE_COPY or
* -D UMM_REALLOC_DEFRAG
*
* Pick one of these two stratagies. UMM_REALLOC_MINIMIZE_COPY grows upward or
* shrinks an allocation, avoiding copy when possible. UMM_REALLOC_DEFRAG gives
* priority with growing the revised allocation toward an adjacent hole in the
* direction of the beginning of the heap when possible.
*
* Status: TODO: These are new options introduced to optionally restore the
* previous defrag propery of realloc. The issue has been raised in the upstream
* repo. No response at this time. Based on response, may propose for upstream.
*/
/*
#define UMM_REALLOC_MINIMIZE_COPY
*/
#define UMM_REALLOC_DEFRAG
/*
* -D UMM_INTEGRITY_CHECK :
*
@ -155,12 +449,28 @@ extern char _heap_start[];
* 4 bytes, so there might be some trailing "extra" bytes which are not checked
* for corruption.
*/
/*
#define UMM_INTEGRITY_CHECK
*/
/*
* -D UMM_POISON :
* Not normally enabled. Full intergity check may exceed 10us.
*/
/*
#define UMM_INTEGRITY_CHECK
*/
#ifdef UMM_INTEGRITY_CHECK
int umm_integrity_check( void );
# define INTEGRITY_CHECK() umm_integrity_check()
extern void umm_corruption(void);
# define UMM_HEAP_CORRUPTION_CB() DBGLOG_FUNCTION( "Heap Corruption!" )
#else
# define INTEGRITY_CHECK() 0
#endif
/////////////////////////////////////////////////
/*
* -D UMM_POISON_CHECK :
* -D UMM_POISON_CHECK_LITE
*
* Enables heap poisoning: add predefined value (poison) before and after each
* allocation, and check before each heap operation that no poison is
@ -185,17 +495,135 @@ extern char _heap_start[];
*
* If poison corruption is detected, the message is printed and user-provided
* callback is called: `UMM_HEAP_CORRUPTION_CB()`
*
* UMM_POISON_CHECK - does a global heap check on all active allocation at
* every alloc API call. May exceed 10us due to critical section with IRQs
* disabled.
*
* UMM_POISON_CHECK_LITE - checks the allocation presented at realloc()
* and free(). Expands the poison check on the current allocation to
* include its nearest allocated neighbors in the heap.
* umm_malloc() will also checks the neighbors of the selected allocation
* before use.
*
* Status: TODO?: UMM_POISON_CHECK_LITE is a new option. We could propose for
* upstream; however, the upstream version has much of the framework for calling
* poison check on each alloc call refactored out. Not sure how this will be
* received.
*/
/*
* Compatibility for deprecated UMM_POISON
*/
#if defined(UMM_POISON) && !defined(UMM_POISON_CHECK)
#define UMM_POISON_CHECK_LITE
#endif
#if defined(DEBUG_ESP_PORT) || defined(DEBUG_ESP_CORE)
#define UMM_POISON
#if !defined(UMM_POISON_CHECK) && !defined(UMM_POISON_CHECK_LITE)
/*
#define UMM_POISON_CHECK
*/
#define UMM_POISON_CHECK_LITE
#endif
#endif
#define UMM_POISON_SIZE_BEFORE 4
#define UMM_POISON_SIZE_AFTER 4
#define UMM_POISON_SIZE_AFTER 4
#define UMM_POISONED_BLOCK_LEN_TYPE uint32_t
#define UMM_HEAP_CORRUPTION_CB() panic()
#if defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE)
void *umm_poison_malloc( size_t size );
void *umm_poison_calloc( size_t num, size_t size );
void *umm_poison_realloc( void *ptr, size_t size );
void umm_poison_free( void *ptr );
int umm_poison_check( void );
// Local Additions to better report location in code of the caller.
void *umm_poison_realloc_fl( void *ptr, size_t size, const char* file, int line );
void umm_poison_free_fl( void *ptr, const char* file, int line );
#if defined(UMM_POISON_CHECK_LITE)
/*
* We can safely do individual poison checks at free and realloc and stay
* under 10us or close.
*/
# define POISON_CHECK() 1
# define POISON_CHECK_NEIGHBORS(c) \
do {\
if(!check_poison_neighbors(c)) \
panic();\
} while(false)
#else
/* Not normally enabled. A full heap poison check may exceed 10us. */
# define POISON_CHECK() umm_poison_check()
# define POISON_CHECK_NEIGHBORS(c) do{}while(false)
#endif
#else
# define POISON_CHECK() 1
# define POISON_CHECK_NEIGHBORS(c) do{}while(false)
#endif
/////////////////////////////////////////////////
#undef DBGLOG_FUNCTION
#undef DBGLOG_FUNCTION_P
#if defined(DEBUG_ESP_PORT) || defined(DEBUG_ESP_OOM) || \
defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE) || \
defined(UMM_INTEGRITY_CHECK)
#define DBGLOG_FUNCTION(fmt, ...) ets_uart_printf(fmt, ##__VA_ARGS__)
#else
#define DBGLOG_FUNCTION(fmt, ...) do { (void)fmt; } while(false)
#endif
/////////////////////////////////////////////////
#if defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE) || defined(UMM_INTEGRITY_CHECK)
#if !defined(DBGLOG_LEVEL) || DBGLOG_LEVEL < 3
// All debug prints in UMM_POISON_CHECK are level 3
#undef DBGLOG_LEVEL
#define DBGLOG_LEVEL 3
#endif
#endif
#if defined(UMM_CRITICAL_METRICS)
struct UMM_TIME_STATS_t {
UMM_TIME_STAT id_malloc;
UMM_TIME_STAT id_realloc;
UMM_TIME_STAT id_free;
#ifdef UMM_INFO
UMM_TIME_STAT id_info;
#endif
#if defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE)
UMM_TIME_STAT id_poison;
#endif
#ifdef UMM_INTEGRITY_CHECK
UMM_TIME_STAT id_integrity;
#endif
UMM_TIME_STAT id_no_tag;
};
#endif
/////////////////////////////////////////////////
#ifdef DEBUG_ESP_OOM
#define MEMLEAK_DEBUG
// umm_*alloc are not renamed to *alloc
// Assumes umm_malloc.h has already been included.
#define umm_zalloc(s) umm_calloc(1,s)
void* malloc_loc (size_t s, const char* file, int line);
void* calloc_loc (size_t n, size_t s, const char* file, int line);
void* realloc_loc (void* p, size_t s, const char* file, int line);
// *alloc are macro calling *alloc_loc calling+checking umm_*alloc()
// they are defined at the bottom of this file
/////////////////////////////////////////////////
#elif defined(UMM_POISON_CHECK)
void* realloc_loc (void* p, size_t s, const char* file, int line);
void free_loc (void* p, const char* file, int line);
#else // !defined(ESP_DEBUG_OOM)
#endif
#ifdef __cplusplus
}
@ -203,12 +631,57 @@ extern char _heap_start[];
#endif /* _UMM_MALLOC_CFG_H */
#ifdef __cplusplus
extern "C" {
#endif
#ifdef DEBUG_ESP_OOM
// this must be outside from "#ifndef _UMM_MALLOC_CFG_H"
// because Arduino.h's <cstdlib> does #undef *alloc
// Arduino.h recall us to redefine them
#include <pgmspace.h>
#define malloc(s) ({ static const char mem_debug_file[] PROGMEM STORE_ATTR = __FILE__; malloc_loc(s, mem_debug_file, __LINE__); })
#define calloc(n,s) ({ static const char mem_debug_file[] PROGMEM STORE_ATTR = __FILE__; calloc_loc(n, s, mem_debug_file, __LINE__); })
#define realloc(p,s) ({ static const char mem_debug_file[] PROGMEM STORE_ATTR = __FILE__; realloc_loc(p, s, mem_debug_file, __LINE__); })
// Reuse pvPort* calls, since they already support passing location information.
void* ICACHE_RAM_ATTR pvPortMalloc(size_t size, const char* file, int line);
void* ICACHE_RAM_ATTR pvPortCalloc(size_t count, size_t size, const char* file, int line);
void* ICACHE_RAM_ATTR pvPortRealloc(void *ptr, size_t size, const char* file, int line);
void* ICACHE_RAM_ATTR pvPortZalloc(size_t size, const char* file, int line);
void ICACHE_RAM_ATTR vPortFree(void *ptr, const char* file, int line);
#define malloc(s) ({ static const char mem_debug_file[] PROGMEM STORE_ATTR = __FILE__; pvPortMalloc(s, mem_debug_file, __LINE__); })
#define calloc(n,s) ({ static const char mem_debug_file[] PROGMEM STORE_ATTR = __FILE__; pvPortCalloc(n, s, mem_debug_file, __LINE__); })
#define realloc(p,s) ({ static const char mem_debug_file[] PROGMEM STORE_ATTR = __FILE__; pvPortRealloc(p, s, mem_debug_file, __LINE__); })
#if defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE)
#define dbg_heap_free(p) ({ static const char mem_debug_file[] PROGMEM STORE_ATTR = __FILE__; vPortFree(p, mem_debug_file, __LINE__); })
#else
#define dbg_heap_free(p) free(p)
#endif
#elif defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE)
#include <pgmspace.h>
void* ICACHE_RAM_ATTR pvPortRealloc(void *ptr, size_t size, const char* file, int line);
#define realloc(p,s) ({ static const char mem_debug_file[] PROGMEM STORE_ATTR = __FILE__; pvPortRealloc(p, s, mem_debug_file, __LINE__); })
void ICACHE_RAM_ATTR vPortFree(void *ptr, const char* file, int line);
//C - to be discussed
/*
Problem, I would like to report the file and line number with the umm poison
event as close as possible to the event. The #define method works for malloc,
calloc, and realloc those names are not as generic as free. A #define free
captures too much. Classes with methods called free are included :(
Inline functions would report the address of the inline function in the .h
not where they are called.
Anybody know a trick to make this work?
Create dbg_heap_free() as an alternative for free() when you need a little
more help in debugging the more challenging problems.
*/
#define dbg_heap_free(p) ({ static const char mem_debug_file[] PROGMEM STORE_ATTR = __FILE__; vPortFree(p, mem_debug_file, __LINE__); })
#else
#define dbg_heap_free(p) free(p)
#endif /* DEBUG_ESP_OOM */
#ifdef __cplusplus
}
#endif

64
cores/esp8266/umm_malloc/umm_performance.cpp
View File

@ -1,64 +0,0 @@
/*
* umm_malloc performance measurments and ESP specifics
*/
#include <stdio.h>
#include <string.h>
#include <pgmspace.h>
#include <core_esp8266_features.h>
#include "umm_performance.h"
#include "umm_stats.h"
extern "C" {
UMM_STATS ummStats = {0, 0, 0, 0};
#ifdef UMM_CRITICAL_PERIOD_ANALYZE
struct _UMM_TIME_STATS time_stats = {
{0xFFFFFFFF, 0U, 0U, 0U},
{0xFFFFFFFF, 0U, 0U, 0U},
{0xFFFFFFFF, 0U, 0U, 0U},
{0xFFFFFFFF, 0U, 0U, 0U} };
bool ICACHE_FLASH_ATTR get_umm_get_perf_data(struct _UMM_TIME_STATS *p, size_t size) {
if (p && sizeof(time_stats) == size) {
uint32_t save_ps = xt_rsil(DEFAULT_CRITICAL_SECTION_INTLEVEL);
memcpy(p, &time_stats, size);
xt_wsr_ps(save_ps);
return true;
}
return false;
}
#endif
#if defined(DEBUG_ESP_PORT) || defined(DEBUG_ESP_ISR)
/*
Printing from the malloc routines is tricky. Since a lot of library calls
will want to do malloc.
Objective: To be able to print "last gasp" diagnostic messages
when interrupts are disabled and w/o availability of heap resources.
*/
int _isr_safe_printf_P(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
int ICACHE_RAM_ATTR _isr_safe_printf_P(const char *fmt, ...) {
/*
To use ets_strlen() and ets_memcpy() safely with PROGMEM, flash storage,
the PROGMEM address must be word (4 bytes) aligned. The destination
address for ets_memcpy must also be word-aligned. We also round the
buf_len up to the nearest word boundary. So that all transfers will be
whole words.
*/
size_t str_len = ets_strlen(fmt);
size_t buf_len = (str_len + 1 + 3) & ~0x03U;
char ram_buf[buf_len] __attribute__ ((aligned(4)));
ets_memcpy(ram_buf, fmt, buf_len);
va_list argPtr;
va_start(argPtr, fmt);
int result = ets_vprintf(ets_uart_putc1, ram_buf, argPtr);
va_end(argPtr);
return result;
}
#endif
};

85
cores/esp8266/umm_malloc/umm_performance.h
View File

@ -1,85 +0,0 @@
/*
* umm_malloc performance measurments and ESP specifics
*/
#ifndef _UMM_PERFORMANCE_H
#define _UMM_PERFORMANCE_H
#ifdef __cplusplus
extern "C" {
#endif
/*
* -D UMM_CRITICAL_PERIOD_ANALYZE :
*
* Build option to collect timing usage data on critical section usage in
* functions: info, malloc, realloc. Collects MIN, MAX, and number of time
* IRQs were disabled at request time. Note, for realloc MAX disabled time
* will not include the time from calling malloc and/or free.
* Examine code for specifics on what info is available and how to access.
*/
// #define UMM_CRITICAL_PERIOD_ANALYZE
/*
Per Devyte, the core currently doesn't support masking a specific interrupt
level. That doesn't mean it can't be implemented, only that at this time
locking is implemented as all or nothing.
https://github.com/esp8266/Arduino/issues/6246#issuecomment-508612609
So for now we default to all, 15.
*/
#ifndef DEFAULT_CRITICAL_SECTION_INTLEVEL
#define DEFAULT_CRITICAL_SECTION_INTLEVEL 15
#endif
#if defined(UMM_CRITICAL_PERIOD_ANALYZE)
// This option adds support for gathering time locked data
typedef struct _TIME_STAT {
uint32_t min;
uint32_t max;
uint32_t start;
uint32_t intlevel;
} time_stat_t;
struct _UMM_TIME_STATS {
time_stat_t id_malloc;
time_stat_t id_realloc;
time_stat_t id_free;
time_stat_t id_info;
};
extern struct _UMM_TIME_STATS time_stats;
bool get_umm_get_perf_data(struct _UMM_TIME_STATS *p, size_t size);
static inline void _critical_entry(time_stat_t *p, uint32_t *saved_ps) {
*saved_ps = xt_rsil(DEFAULT_CRITICAL_SECTION_INTLEVEL);
if (0U != (*saved_ps & 0x0FU)) {
p->intlevel += 1U;
}
p->start = esp_get_cycle_count();
}
static inline void _critical_exit(time_stat_t *p, uint32_t *saved_ps) {
uint32_t elapse = esp_get_cycle_count() - p->start;
if (elapse < p->min)
p->min = elapse;
if (elapse > p->max)
p->max = elapse;
xt_wsr_ps(*saved_ps);
}
#endif
#if defined(DEBUG_ESP_PORT) || defined(DEBUG_ESP_ISR)
int _isr_safe_printf_P(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
#endif
#ifdef __cplusplus
}
#endif
#endif /* _UMM_PERFORMANCE_H */

241
cores/esp8266/umm_malloc/umm_poison.c Normal file
View File

@ -0,0 +1,241 @@
#if defined(BUILD_UMM_MALLOC_C)
/* poisoning (UMM_POISON_CHECK) {{{ */
#if defined(UMM_POISON_CHECK) || defined(UMM_POISON_CHECK_LITE)
#define POISON_BYTE (0xa5)
/*
* Yields a size of the poison for the block of size `s`.
* If `s` is 0, returns 0.
*/
static size_t poison_size(size_t s) {
return(s ? (UMM_POISON_SIZE_BEFORE +
sizeof(UMM_POISONED_BLOCK_LEN_TYPE) +
UMM_POISON_SIZE_AFTER)
: 0);
}
/*
* Print memory contents starting from given `ptr`
*/
static void dump_mem ( const unsigned char *ptr, size_t len ) {
while (len--) {
DBGLOG_ERROR(" 0x%.2x", (unsigned int)(*ptr++));
}
}
/*
* Put poison data at given `ptr` and `poison_size`
*/
static void put_poison( unsigned char *ptr, size_t poison_size ) {
memset(ptr, POISON_BYTE, poison_size);
}
/*
* Check poison data at given `ptr` and `poison_size`. `where` is a pointer to
* a string, either "before" or "after", meaning, before or after the block.
*
* If poison is there, returns 1.
* Otherwise, prints the appropriate message, and returns 0.
*/
static int check_poison( const unsigned char *ptr, size_t poison_size,
const char *where) {
size_t i;
int ok = 1;
for (i = 0; i < poison_size; i++) {
if (ptr[i] != POISON_BYTE) {
ok = 0;
break;
}
}
if (!ok) {
DBGLOG_ERROR( "No poison %s block at: 0x%lx, actual data:", where, (unsigned long)ptr);
dump_mem(ptr, poison_size);
DBGLOG_ERROR( "\n" );
}
return ok;
}
/*
* Check if a block is properly poisoned. Must be called only for non-free
* blocks.
*/
static int check_poison_block( umm_block *pblock ) {
int ok = 1;
if (pblock->header.used.next & UMM_FREELIST_MASK) {
DBGLOG_ERROR( "check_poison_block is called for free block 0x%lx\n", (unsigned long)pblock);
} else {
/* the block is used; let's check poison */
unsigned char *pc = (unsigned char *)pblock->body.data;
unsigned char *pc_cur;
pc_cur = pc + sizeof(UMM_POISONED_BLOCK_LEN_TYPE);
if (!check_poison(pc_cur, UMM_POISON_SIZE_BEFORE, "before")) {
ok = 0;
goto clean;
}
pc_cur = pc + *((UMM_POISONED_BLOCK_LEN_TYPE *)pc) - UMM_POISON_SIZE_AFTER;
if (!check_poison(pc_cur, UMM_POISON_SIZE_AFTER, "after")) {
ok = 0;
goto clean;
}
}
clean:
return ok;
}
/*
* Takes a pointer returned by actual allocator function (`umm_malloc` or
* `umm_realloc`), puts appropriate poison, and returns adjusted pointer that
* should be returned to the user.
*
* `size_w_poison` is a size of the whole block, including a poison.
*/
static void *get_poisoned( void *v_ptr, size_t size_w_poison ) {
unsigned char *ptr = (unsigned char *)v_ptr;
if (size_w_poison != 0 && ptr != NULL) {
/* Poison beginning and the end of the allocated chunk */
put_poison(ptr + sizeof(UMM_POISONED_BLOCK_LEN_TYPE),
UMM_POISON_SIZE_BEFORE);
put_poison(ptr + size_w_poison - UMM_POISON_SIZE_AFTER,
UMM_POISON_SIZE_AFTER);
/* Put exact length of the user's chunk of memory */
*(UMM_POISONED_BLOCK_LEN_TYPE *)ptr = (UMM_POISONED_BLOCK_LEN_TYPE)size_w_poison;
/* Return pointer at the first non-poisoned byte */
ptr += sizeof(UMM_POISONED_BLOCK_LEN_TYPE) + UMM_POISON_SIZE_BEFORE;
}
return (void *)ptr;
}
/*
* Takes "poisoned" pointer (i.e. pointer returned from `get_poisoned()`),
* and checks that the poison of this particular block is still there.
*
* Returns unpoisoned pointer, i.e. actual pointer to the allocated memory.
*/
static void *get_unpoisoned( void *v_ptr ) {
unsigned char *ptr = (unsigned char *)v_ptr;
if (ptr != NULL) {
unsigned short int c;
ptr -= (sizeof(UMM_POISONED_BLOCK_LEN_TYPE) + UMM_POISON_SIZE_BEFORE);
/* Figure out which block we're in. Note the use of truncated division... */
c = (((char *)ptr)-(char *)(&(umm_heap[0])))/sizeof(umm_block);
check_poison_block(&UMM_BLOCK(c));
}
return (void *)ptr;
}
/* }}} */
/* ------------------------------------------------------------------------ */
void *umm_poison_malloc( size_t size ) {
void *ret;
size += poison_size(size);
ret = umm_malloc( size );
ret = get_poisoned(ret, size);
return ret;
}
/* ------------------------------------------------------------------------ */
void *umm_poison_calloc( size_t num, size_t item_size ) {
void *ret;
size_t size = item_size * num;
size += poison_size(size);
ret = umm_malloc(size);
if (NULL != ret)
memset(ret, 0x00, size);
ret = get_poisoned(ret, size);
return ret;
}
/* ------------------------------------------------------------------------ */
void *umm_poison_realloc( void *ptr, size_t size ) {
void *ret;
ptr = get_unpoisoned(ptr);
size += poison_size(size);
ret = umm_realloc( ptr, size );
ret = get_poisoned(ret, size);
return ret;
}
/* ------------------------------------------------------------------------ */
void umm_poison_free( void *ptr ) {
ptr = get_unpoisoned(ptr);
umm_free( ptr );
}
/*
* Iterates through all blocks in the heap, and checks poison for all used
* blocks.
*/
int umm_poison_check(void) {
UMM_CRITICAL_DECL(id_poison);
int ok = 1;
unsigned short int cur;
if (umm_heap == NULL) {
umm_init();
}
UMM_CRITICAL_ENTRY(id_poison);
/* Now iterate through the blocks list */
cur = UMM_NBLOCK(0) & UMM_BLOCKNO_MASK;
while( UMM_NBLOCK(cur) & UMM_BLOCKNO_MASK ) {
if ( !(UMM_NBLOCK(cur) & UMM_FREELIST_MASK) ) {
/* This is a used block (not free), so, check its poison */
ok = check_poison_block(&UMM_BLOCK(cur));
if (!ok){
break;
}
}
cur = UMM_NBLOCK(cur) & UMM_BLOCKNO_MASK;
}
UMM_CRITICAL_EXIT(id_poison);
return ok;
}
/* ------------------------------------------------------------------------ */
#endif
#endif // defined(BUILD_UMM_MALLOC_C)

36
cores/esp8266/umm_malloc/umm_stats.h
View File

@ -1,36 +0,0 @@
/*
* umm_malloc heap statistics
*/
#ifndef _UMM_STATS_H
#define _UMM_STATS_H
#ifdef __cplusplus
extern "C" {
#endif
typedef struct UMM_STATS_t {
unsigned short int free_blocks;
unsigned short int free_blocks_min;
size_t alloc_max_size;
size_t oom_count;
} UMM_STATS;
extern UMM_STATS ummStats;
size_t ICACHE_FLASH_ATTR umm_free_heap_size_min( void );
size_t ICACHE_FLASH_ATTR umm_free_heap_size_min_reset( void );
inline size_t umm_get_max_alloc_size( void ) {
return ummStats.alloc_max_size;
}
inline size_t umm_get_oom_count( void ) {
return ummStats.oom_count;
}
#ifdef __cplusplus
}
#endif
#endif /* _UMM_STATS_H */