mirror of
https://sourceware.org/git/glibc.git
synced 2025-08-01 10:06:57 +03:00
3e8814903c
The current approach tracks math maximum supported errors by explicitly setting them per function and architecture. On newer implementations or new compiler versions, the file is updated with newer values if it shows higher results. The idea is to track the maximum known error, to update the manual with the obtained values. The constant libm-test-ulps shows little value, where it is usually a mechanical change done by the maintainer, for past releases it is usually ignored whether the ulp change resulted from a compiler regression, and the math tests already have a maximum ulp error that triggers a regression. It was shown by a recent update after the new acosf [1] implementation that is correctly rounded, where the libm-test-ulps was indeed from a compiler issue. This patch removes all arch-specific libm-test-ulps, adds system generic libm-test-ulps where applicable, and changes its semantics. The generic files now track specific implementation constraints, like if it is expected to be correctly rounded, or if the system-specific has different error expectations. Now multiple libm-test-ulps can be defined, and system-specific overrides generic implementation. This is for the case where arch-specific implementation might show worse precision than generic implementation, for instance, the cbrtf on i686. Regressions are only reported if the implementation shows larger errors than 9 ulps (13 for IBM long double) unless it is overridden by libm-test-ulps and the maximum error is not printed at the end of tests. The regen-ulps rule is also removed since it does not make sense to update the libm-test-ulps automatically. The manual error table is also removed, Paul Zimmermann and others have been tracking libm precision with a more comprehensive analysis for some releases; so link to his work instead. [1] https://sourceware.org/git/?p=glibc.git;a=commit;h=9cc9f8e11e8fb8f54f1e84d9f024917634a78201
157 lines
6.7 KiB
Plaintext
157 lines
6.7 KiB
Plaintext
README for libm-test math test suite
|
|
====================================
|
|
|
|
The libm-test math test suite tests a number of function points of
|
|
math functions in the GNU C library. The following sections contain a
|
|
brief overview. Please note that the test drivers and the Python
|
|
script "gen-libm-test.py" have some options. A full list of options
|
|
is available with --help (for the test drivers) and -h for
|
|
"gen-libm-test.py".
|
|
|
|
|
|
What is tested?
|
|
===============
|
|
The tests just evaluate the functions at specified points and compare
|
|
the results with precomputed values and the requirements of the ISO
|
|
C99 standard.
|
|
|
|
Besides testing the special values mandated by IEEE 754 (infinities,
|
|
NaNs and minus zero), some more or less random values are tested.
|
|
|
|
Files that are part of libm-test
|
|
================================
|
|
|
|
The main files are "libm-test-<func>.inc". They are independent of
|
|
the target platform and the specific real floating type and format and
|
|
contain placeholder test "templates" for math functions defined in
|
|
libm. These files, along with generated files named
|
|
"auto-libm-test-out-<func>", are preprocessed by the Python script
|
|
"gen-libm-test.py" to expand the templates and produce a set of test
|
|
cases for each math function that are specific to the target platform
|
|
but still independent of the real floating type. The results of the
|
|
processing are "libm-test-<func>.c" and a file "libm-test-ulps.h" with
|
|
specific math results that can be either generic for the floating
|
|
type or platform specific.
|
|
|
|
The test drivers "test-double-<func>.c", "test-float-<func>.c", and
|
|
"test-ldouble-<func>.c", generated by the Makefile, test the normal
|
|
double, float and long double implementation of libm. Each driver
|
|
selects the desired real floating type to exercise the math functions
|
|
to test with (float, double, or long double) by defining a small set
|
|
of macros just before including the generic "libm-test.c" file. Each
|
|
driver is compiled into a single executable test program with the
|
|
corresponding name.
|
|
|
|
The math tests do not report up to 9 Units of Least Precision (ULP)
|
|
(13 for IBM long double format) difference between the obtained
|
|
result and the expected one as a regression. The "gen-libm-test.py"
|
|
script looks for files named "libm-test-ulps" in the sysdep directories
|
|
to generate the "libm-test-ulps.h" file.
|
|
|
|
The "auto-libm-test-out-<func>" files contain sets of test cases to
|
|
exercise, the conditions under which to exercise each, and the
|
|
expected results. The files are generated by the
|
|
"gen-auto-libm-tests" program from the "auto-libm-test-in" file. See
|
|
the comments in gen-auto-libm-tests.c for details about the content
|
|
and format of the -in and -out files.
|
|
|
|
How can I use "libm-test-ulps"?
|
|
====================================
|
|
|
|
A "libm-test-ulps" is required only to test for extra constraints in
|
|
the math tests. The file contains lines for maximal errors of single
|
|
functions, like:
|
|
|
|
Function "yn":
|
|
float: 2
|
|
double: 6
|
|
|
|
It means that if the "yn" shows error larger than 2 ULP for float
|
|
or 6 ULP for double, the related test for "symbol" will fail. It can
|
|
be useful to check for correctly rounded implementation, where the
|
|
expected ULP is 0.
|
|
|
|
The function is tested with default FE_TONEAREST rounding mode. To
|
|
check with a different one, the function definition name should be
|
|
prepended with an underline plus the rounding mode 'downward' (FE_DOWNWARD),
|
|
'towardzero' (FE_TOWARDZERO), or 'upward' (FE_UPWARD). For instance,
|
|
|
|
Function "yn_downward":
|
|
float: 3
|
|
double: 7
|
|
|
|
It means that 'yn' will be checked with FE_DOWNWARD rounding mode
|
|
and any error larger than 3 ULPs for float or 7 ULPs for double will be
|
|
reported as a regression.
|
|
|
|
The keywords are float, double, ldouble, and float128.
|
|
|
|
Also, multiple "libm-test-ulps" can be added, "gen-libm-test.py" will
|
|
merge the input in only one table.
|
|
|
|
Note that the test drivers have an option "-u" to output an unsorted
|
|
list of all epsilons that the functions have. The output can be read
|
|
in directly but it's better to pretty print it first.
|
|
"gen-libm-test.py" has an option to generate a pretty-printed and
|
|
sorted new ULPs file from the output of the test drivers.
|
|
|
|
|
|
Adding tests to libm-test-<func>.inc
|
|
====================================
|
|
|
|
The tests are evaluated by a set of special test macros. The macros
|
|
start with "TEST_" followed by a specification the input values, an
|
|
underscore and a specification of the output values. As an example,
|
|
the test macro for a function with input of type FLOAT (FLOAT is
|
|
either float, double, long double) and output of type FLOAT is
|
|
"TEST_f_f". The macro's parameter are the name of the function, the
|
|
input parameter, output parameter and optionally one exception
|
|
parameter.
|
|
|
|
The accepted parameter types are:
|
|
- "f" for FLOAT
|
|
- "j" for long double.
|
|
- "a" for ARG_FLOAT, the argument type for narrowing functions.
|
|
- "b" for boolean - just tests if the output parameter evaluates to 0
|
|
or 1 (only for output).
|
|
- "c" for complex. This parameter needs two values, first the real,
|
|
then the imaginary part.
|
|
- "i" for int.
|
|
- "l" for long int.
|
|
- "L" for long long int.
|
|
- "u" for unsigned int.
|
|
- "M" for intmax_t.
|
|
- "U" for uintmax_t.
|
|
- "p" for an argument (described in the previous character) passed
|
|
through a pointer rather than directly.
|
|
- "F" for the address of a FLOAT (only as input parameter)
|
|
- "I" for the address of an int (only as input parameter)
|
|
- "1" for an additional output (either output through a pointer passed
|
|
as an argument, or to a global variable such as signgam).
|
|
|
|
How to read the test output
|
|
===========================
|
|
|
|
Running each test on its own at the default level of verbosity will
|
|
print on stdout a line describing the implementation of math functions
|
|
exercised by the test (float, double, or long double). This is then
|
|
followed by the details of test failures (if any). The output concludes
|
|
by a summary listing the number of test cases exercised and the number
|
|
of test failures uncovered.
|
|
|
|
For each test failure (and for each test case at higher levels of
|
|
verbosity), the output contains the name of the function under test
|
|
and its arguments or conditions that triggered the failure. Note
|
|
that the name of the function in the output need not correspond
|
|
exactly to the name of the math function actually invoked. For example,
|
|
the output will refer to the "acos" function even if the actual function
|
|
under test is acosf (for the float version) or acosl (for the long
|
|
double version). Also note that the function arguments may be shown
|
|
in either the decimal or the hexadecimal floating point format which
|
|
may or may not correspond to the format used in the auto-libm-test-in
|
|
file. Besides the name of the function, for each test failure the
|
|
output contains the actual and expected results and the difference
|
|
between the two, printed in both the decimal and hexadecimal
|
|
floating point format, and the ULP and maximum ULP for the test
|
|
case.
|