database/mariadb-columnstore-engine
1
0
Fork 0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-07-29 08:21:15 +03:00
Code Activity

MCOL-4738 AVG() returns a wrong result

Browse Source 
On AMD64 machines, the fpu is 80 bits. The unused bits must be masked for memcmp to work properly. For other archetectures, we don't want to mask those bits.
This commit is contained in:
David Hall
2021-07-06 19:50:00 -05:00
parent b9bd207d3b
commit 8332ab8974
3 changed files with 25 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
utils/rowgroup/rowgroup.h
View File

@ -1369,12 +1369,10 @@ inline void Row::setFloatField(float val, uint32_t colIndex)
inline void Row::setLongDoubleField(const long double& val, uint32_t colIndex)
{
uint8_t* p = &data[offsets[colIndex]];
*((long double*)p) = val;
if (sizeof(long double) == 16)
{
// zero out the unused portion as there may be garbage there.
*((uint64_t*)p+1) &= 0x000000000000FFFFULL;
}
*reinterpret_cast<long double*>(p) = val;
#ifdef MASK_LONGDOUBLE
*(reinterpret_cast<uint64_t*>(p)+1) &= 0x000000000000FFFFULL;
#endif
}
inline void Row::setInt128Field(const int128_t& val, uint32_t colIndex)