Use a proper C tokenizer to implement the obsolete typedefs test.

The test for obsolete typedefs in installed headers was implemented
using grep, and could therefore get false positives on e.g. “ulong”
in a comment.  It was also scanning all of the headers included by
our headers, and therefore testing headers we don’t control, e.g.
Linux kernel headers.

This patch splits the obsolete-typedef test from
scripts/check-installed-headers.sh to a separate program,
scripts/check-obsolete-constructs.py.  Being implemented in Python,
it is feasible to make it tokenize C accurately enough to avoid false
positives on the contents of comments and strings.  It also only
examines $(headers) in each subdirectory--all the headers we install,
but not any external dependencies of those headers.  Headers whose
installed name starts with finclude/ are ignored, on the assumption
that they contain Fortran.

It is also feasible to make the new test understand the difference
between _defining_ the obsolete typedefs and _using_ the obsolete
typedefs, which means posix/{bits,sys}/types.h no longer need to be
exempted.  This uncovered an actual bug in bits/types.h: __quad_t and
__u_quad_t were being used to define __S64_TYPE, __U64_TYPE,
__SQUAD_TYPE and __UQUAD_TYPE.  These are changed to __int64_t and
__uint64_t respectively.  This is a safe change, despite the comments
in bits/types.h claiming a difference between __quad_t and __int64_t,
because those comments are incorrect.  In all current ABIs, both
__quad_t and __int64_t are ‘long’ when ‘long’ is a 64-bit type, and
‘long long’ when ‘long’ is a 32-bit type, and similarly for __u_quad_t
and __uint64_t.  (Changing the types to be what the comments say they
are would be an ABI break, as it affects C++ name mangling.)  This
patch includes a minimal change to make the comments not completely
wrong.

sys/types.h was defining the legacy BSD u_intN_t typedefs using a
construct that was not necessarily consistent with how the C99 uintN_t
typedefs are defined, and is also too complicated for the new script to
understand (it lexes C relatively accurately, but it does not attempt
to expand preprocessor macros, nor does it do any actual parsing).
This patch cuts all of that out and uses bits/types.h's __uintN_t typedefs
to define u_intN_t instead.  This is verified to not change the ABI on
any supported architecture, via the c++-types test, which means u_intN_t
and uintN_t were, in fact, consistent on all supported architectures.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

	* scripts/check-obsolete-constructs.py: New test script.
	* scripts/check-installed-headers.sh: Remove tests for
	obsolete typedefs, superseded by check-obsolete-constructs.py.
	* Rules: Run scripts/check-obsolete-constructs.py over $(headers)
	as a special test.  Update commentary.
	* posix/bits/types.h (__SQUAD_TYPE, __S64_TYPE): Define as __int64_t.
	(__UQUAD_TYPE, __U64_TYPE): Define as __uint64_t.
	Update commentary.
	* posix/sys/types.h (__u_intN_t): Remove.
	(u_int8_t): Typedef using __uint8_t.
	(u_int16_t): Typedef using __uint16_t.
	(u_int32_t): Typedef using __uint32_t.
	(u_int64_t): Typedef using __uint64_t.

scripts/check-installed-headers.sh

@@ -16,11 +16,9 @@
 # License along with the GNU C Library; if not, see
 # <http://www.gnu.org/licenses/>.
 
-# Check installed headers for cleanliness.  For each header, confirm
-# that it's possible to compile a file that includes that header and
-# does nothing else, in several different compilation modes.  Also,
-# scan the header for a set of obsolete typedefs that should no longer
-# appear.
+# For each installed header, confirm that it's possible to compile a
+# file that includes that header and does nothing else, in several
+# different compilation modes.
 
 # These compilation switches assume GCC or compatible, which is probably
 # fine since we also assume that when _building_ glibc.
@@ -31,13 +29,6 @@ cxx_modes="-std=c++98 -std=gnu++98 -std=c++11 -std=gnu++11"
 # These are probably the most commonly used three.
 lib_modes="-D_DEFAULT_SOURCE=1 -D_GNU_SOURCE=1 -D_XOPEN_SOURCE=700"
 
-# sys/types.h+bits/types.h have to define the obsolete types.
-# rpc(svc)/* have the obsolete types too deeply embedded in their API
-# to remove.
-skip_obsolete_type_check='*/sys/types.h|*/bits/types.h|*/rpc/*|*/rpcsvc/*'
-obsolete_type_re=\
-'\<((__)?(quad_t|u(short|int|long|_(char|short|int([0-9]+_t)?|long|quad_t))))\>'
-
 if [ $# -lt 3 ]; then
     echo "usage: $0 c|c++ \"compile command\" header header header..." >&2
     exit 2
@@ -46,14 +37,10 @@ case "$1" in
     (c)
         lang_modes="$c_modes"
         cih_test_c=$(mktemp ${TMPDIR-/tmp}/cih_test_XXXXXX.c)
-        already="$skip_obsolete_type_check"
     ;;
     (c++)
         lang_modes="$cxx_modes"
         cih_test_c=$(mktemp ${TMPDIR-/tmp}/cih_test_XXXXXX.cc)
-        # The obsolete-type check can be skipped for C++; it is
-        # sufficient to do it for C.
-        already="*"
     ;;
     (*)
         echo "usage: $0 c|c++ \"compile command\" header header header..." >&2
@@ -155,22 +142,8 @@ $expanded_lib_mode
 int avoid_empty_translation_unit;
 EOF
             if $cc_cmd -fsyntax-only $lang_mode "$cih_test_c" 2>&1
-            then
-                includes=$($cc_cmd -fsyntax-only -H $lang_mode \
-                              "$cih_test_c" 2>&1 | sed -ne 's/^[.][.]* //p')
-                for h in $includes; do
-                    # Don't repeat work.
-                    eval 'case "$h" in ('"$already"') continue;; esac'
-
-                    if grep -qE "$obsolete_type_re" "$h"; then
-                        echo "*** Obsolete types detected:"
-                        grep -HE "$obsolete_type_re" "$h"
-                        failed=1
-                    fi
-                    already="$already|$h"
-                done
-            else
-                failed=1
+            then :
+            else failed=1
             fi
         done
     done