database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-06-27 23:21:58 +03:00
Code Activity

Fix corner-case loss of precision in numeric_power().

Browse Source 
This fixes a loss of precision that occurs when the first input is
very close to 1, so that its logarithm is very small.

Formerly, during the initial low-precision calculation to estimate the
result weight, the logarithm was computed to a local rscale that was
capped to NUMERIC_MAX_DISPLAY_SCALE (1000). However, the base may be
as close as 1e-16383 to 1, hence its logarithm may be as small as
1e-16383, and so the local rscale needs to be allowed to exceed 16383,
otherwise all precision is lost, leading to a poor choice of rscale
for the full-precision calculation.

Fix this by removing the cap on the local rscale during the initial
low-precision calculation, as we already do in the full-precision
calculation. This doesn't change the fact that the initial calculation
is a low-precision approximation, computing the logarithm to around 8
significant digits, which is very fast, especially when the base is
very close to 1.

Patch by me, reviewed by Alvaro Herrera.

Discussion: https://postgr.es/m/CAEZATCV-Ceu%2BHpRMf416yUe4KKFv%3DtdgXQAe5-7S9tD%3D5E-T1g%40mail.gmail.com
This commit is contained in:
Dean Rasheed
2021-10-06 13:16:51 +01:00
parent ba216d3b54
commit e54a758d24
3 changed files with 12 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
src/backend/utils/adt/numeric.c
View File

@ -10266,9 +10266,13 @@ power_var(const NumericVar *base, const NumericVar *exp, NumericVar *result)
*/
ln_dweight = estimate_ln_dweight(base);
/*
* Set the scale for the low-precision calculation, computing ln(base) to
* around 8 significant digits. Note that ln_dweight may be as small as
* -SHRT_MAX, so the scale may exceed NUMERIC_MAX_DISPLAY_SCALE here.
*/
local_rscale = 8 - ln_dweight;
local_rscale = Max(local_rscale, NUMERIC_MIN_DISPLAY_SCALE);
local_rscale = Min(local_rscale, NUMERIC_MAX_DISPLAY_SCALE);
ln_var(base, &ln_base, local_rscale);