* Fix String.replace overlapping strcpy
Fixes#5949
Adds a test from the issue above and fixes the problem valgrind found.
Additional pathological memcpy->memmove fixes
* Fix String::replace()
Fixes#5883 and supercedes #5890
The replace() function was using len() while in the middle of buffer
operations. In SSO mode len() is not stored separately and is a call to
strlen(), which may not be legal if you're in the middle of overwriting
the SSO buffer, as was the case in ::replace when the replacement string
was longer than the find string. This caused potential garbage at the
end of the string when accessed. Instead, just cache the length in a
local while doing the operation.
Add in test cases from #5890 as well as some new ones that fail on the
unmodified core.
* Fix stack smashing error on 64b
When pointers are 8 bytes long, the size of a String is larger than 16
chars. Increase the allocated array we're using in the test to avoid a
"stack smashing" error.
* Manually call destructor in test
Just for clarity, manually call the destructor for the Strings() that
are "placement new'd" in the String tests. It is a no-op for the
existing test, since thanks to SSO there are no memory allocations, but
will help in case someone adds tests later which include longer strings.
Reduce String memory overhead from 24 bytes to 16 bytes by limiting the
maximum string length to <64Kbytes (which is larger than heap so no
effective problem).
Add Small String Optimization, SSO, which instead of allocating pointers
to small strings on the heap will store the string in place of the
pointer in the class. This should reduce memory fragmentation as
Save up to 12 chars (11 + \0) in String itself by using the terminating
\0 in the inline string as a flag to identify if this is a SSO or a heap
string.
Add a host test that verifies that no memory is allocated until a
full 11 characters are assigned to a string, as well as checking all
intermediate values.
No user code changes should be required to work with this optimization.
As @devyte noticed, PR #4955 has an issue when you catenate a string to
itself and the string used to hold a longer value because it does not
explicitly 0-terminate the resulting string. If the string was extended,
however, reserve() would 0-terminate by default.
Always terminate the result of `s += s;` now.
When a string is concatted to itself, the pointer to its c_str can change
due to realloc(). This would invalidate the passed-in pointer being
concatted, and cause a use-after-free error. Special case this to avoid
the issue. Now "a += a;" works properly.
Also use sprintf(%{l}d) instead of non-POSIX ltoa/itoa calls to construct a
string from a signed number (in base 10 only). The non-posix versions don't
handle INT_MIN properly on either host_tests or on the ESP8266.
memcpy() is undefined when source and destination overlap. String::trim
uses it when shifting the string left to remove left padding.
Replace with memmove() which is always safe, even when overlapped.
MD5Builder tests have been randomly, non-repeatably failing due to a problem
with the returned value of MD5Builder.
Valgrind detected a strncpy with an overlapping memory range, which is
an undefined operation. Fix it with a memmove instead, and get rid
of a couple #define redefinitions which were causing compile warnings
on the host side as well.
* Added constant time strings comparison to avoid possible time-based attacks
* Fixed data types
* Fixed indentation
* Moved string comnparison in constant time to String class; modified function body to assure constant time comparison despite compiler optimizations
* Removed wrong code
* Fixed error and prevented compiler optimization to delete u1 local variable
* Avoid timing attacks on string comparison
* Minor
* changed counter names, removed else
realloc() is called with newSize > 0 (at least 16), so newbuffer==0 means the old memory was not deallocated. Therefore, the pointer should still point to the old buffer. This change should resolve issue #3516.
Calling String::reserve() causes a crash if String object was in invalidated state. Per the comment on the method's declaration in ESP_SSD1306.h, This method was supposed to recover invalidated strings. This change fixes the edge case bug in String::changeBuffer() which is the root cause of the crash exposed from String::reserve().
Following test code was used to reproduce the problem and also to validate the fix:
String result;
while(true){
char c = 'A';
result += c; // the loop will cause malloc() to fail at some point.
if (result.c_str()==0)
{
Serial.println("String INVALIDATED!!!!!");
result.reserve(0); // before fix, this would crash.
Serial.println("Trying to empty....");
result="";
Serial.println("Emptied!!!!");
break;
}
}
PSTR() and F() macros correctly place string into flash memory relying
on PROGMEM
PROGMEM uses ICACHE_RODATA_ATTR
Print and String classes fixed up
str* classes fixed up
This includes empty stubs for most core Arduino functions.
Need to actually implement all those digital reads writes whatever.
Need to prebuild toolchains (xtensa-elf-lx106) for 3 platforms and put them on some download server.
Need to do the same with esptool.
Need to fix 0x40000 binary generation and add correct upload commands.
Maybe even implement uploads over WiFi.
