From 5b4ecd3f95695ef593e4474b4ab5a117291ba5fc Mon Sep 17 00:00:00 2001 From: Joseph Myers Date: Wed, 24 Feb 2016 17:15:12 +0000 Subject: Require Linux 3.2 except on x86 / x86_64, 3.2 headers everywhere. In I proposed a minimum Linux kernel version of 3.2 for glibc 2.24, since Linux 2.6.32 has reached EOL. In the discussion in February, some concerns were expressed about compatibility with OpenVZ containers. It's not clear that these are real issues, given OpenVZ backporting kernel features and faking the kernel version for guest software, as discussed in . It's also not clear that supporting running GNU/Linux distributions from late 2016 (at the earliest) on a kernel series from 2009 is a sensible expectation. However, as an interim step, this patch increases the requirement everywhere except x86 / x86_64 (since the controversy was only about those architectures); the special caveats and settings can easily be removed later when we're ready to increase the requirements on x86 / x86_64 (and if someone would like to raise the issue on LWN as suggested in the previous discussion, that would be welcome). 3.2 kernel headers are required everywhere by this patch. (x32 already requires 3.4 or later, so is unaffected by this patch.) As usual for such a change, this patch only changes the configure scripts and associated documentation. The intent is to follow up with removal of dead __LINUX_KERNEL_VERSION conditionals. Each __ASSUME_* or other macro that becomes dead can then be removed independently. Tested for x86_64 and x86. * sysdeps/unix/sysv/linux/configure.ac (LIBC_LINUX_VERSION): Define to 3.2.0. (arch_minimum_kernel): Likewise. * sysdeps/unix/sysv/linux/configure: Regenerated. * sysdeps/unix/sysv/linux/i386/configure.ac (arch_minimum_kernel): Define to 2.6.32. * sysdeps/unix/sysv/linux/i386/configure: Regenerated. * sysdeps/unix/sysv/linux/x86_64/64/configure.ac (arch_minimum_kernel): Define to 2.6.32. * sysdeps/unix/sysv/linux/x86_64/64/configure: Regenerated. * README: Document Linux 3.2 requirement. * manual/install.texi (Linux): Document Linux 3.2 headers requirement. * INSTALL: Regenerated. --- INSTALL | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'INSTALL') diff --git a/INSTALL b/INSTALL index a0d0c058cd..31e256d413 100644 --- a/INSTALL +++ b/INSTALL @@ -426,7 +426,7 @@ Specific advice for GNU/Linux systems ===================================== If you are installing the GNU C Library on GNU/Linux systems, you need -to have the header files from a 2.6.32 or newer kernel around for +to have the header files from a 3.2 or newer kernel around for reference. These headers must be installed using 'make headers_install'; the headers present in the kernel source directory are not suitable for direct use by the GNU C Library. You do not need to -- cgit v1.2.3 From b9f5c3acc01a836369b9cbc3fc50ca550261a185 Mon Sep 17 00:00:00 2001 From: Paul Pluzhnikov Date: Sun, 5 Jun 2016 08:41:13 -0700 Subject: 2016-06-05 Paul Pluzhnikov * manual/install.texi: Remove mention of --without-tls * INSTALL: Regenerate. --- ChangeLog | 5 +++++ INSTALL | 6 ------ manual/install.texi | 6 ------ 3 files changed, 5 insertions(+), 12 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index be1e0bb759..84738f52ea 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2016-06-05 Paul Pluzhnikov + + * manual/install.texi: Remove mention of --without-tls + * INSTALL: Regenerate. + 2016-06-05 Florian Weimer * dlfcn/tst-rec-dlopen.c (custom_malloc_hook, old_malloc_hook): diff --git a/INSTALL b/INSTALL index 31e256d413..ec3445f2d6 100644 --- a/INSTALL +++ b/INSTALL @@ -116,12 +116,6 @@ will be used, and CFLAGS sets optimization options for the compiler. program linked statically with the NSS libraries cannot be dynamically reconfigured to use a different name database. -'--without-tls' - By default the C library is built with support for thread-local - storage if the used tools support it. By using '--without-tls' - this can be prevented though there generally is no reason since it - creates compatibility problems. - '--enable-hardcoded-path-in-tests' By default, dynamic tests are linked to run with the installed C library. This option hardcodes the newly built C library path in diff --git a/manual/install.texi b/manual/install.texi index 95021b4a50..79ee45fcdb 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -146,12 +146,6 @@ This is not recommended because it defeats the purpose of NSS; a program linked statically with the NSS libraries cannot be dynamically reconfigured to use a different name database. -@item --without-tls -By default the C library is built with support for thread-local storage -if the used tools support it. By using @samp{--without-tls} this can be -prevented though there generally is no reason since it creates -compatibility problems. - @item --enable-hardcoded-path-in-tests By default, dynamic tests are linked to run with the installed C library. This option hardcodes the newly built C library path in dynamic tests -- cgit v1.2.3 From b55a155f005366b5a6c886a2e4c57707b341b619 Mon Sep 17 00:00:00 2001 From: Rical Jasan Date: Thu, 6 Oct 2016 12:28:45 +0530 Subject: Manual typos: Installing 2016-05-06 Rical Jasan * manual/install.texi: Fix typos in the manual. * INSTALL: Regenerated. --- ChangeLog | 3 +++ INSTALL | 8 ++++---- manual/install.texi | 6 +++--- 3 files changed, 10 insertions(+), 7 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 78ee06d6d7..8b91b32f15 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,8 @@ 2016-10-06 Rical Jasan + * manual/install.texi: Fix typos in the manual. + * INSTALL: Regenerated. + * manual/lang.texi: Fix typos in the manual. * manual/probes.texi: Fix typos in the manual. diff --git a/INSTALL b/INSTALL index ec3445f2d6..51f26bf962 100644 --- a/INSTALL +++ b/INSTALL @@ -155,8 +155,8 @@ will be used, and CFLAGS sets optimization options for the compiler. can configure with '--disable-werror'. '--disable-mathvec' - By default for x86_64, the GNU C Library is built with vector math - library. Use this option to disable vector math library. + By default for x86_64, the GNU C Library is built with the vector + math library. Use this option to disable the vector math library. '--build=BUILD-SYSTEM' '--host=HOST-SYSTEM' @@ -193,8 +193,8 @@ will be used, and CFLAGS sets optimization options for the compiler. To build the library and related programs, type 'make'. This will produce a lot of output, some of which may look like errors from 'make' -but isn't. Look for error messages from 'make' containing '***'. Those -indicate that something is seriously wrong. +but aren't. Look for error messages from 'make' containing '***'. +Those indicate that something is seriously wrong. The compilation process can take a long time, depending on the configuration and the speed of your machine. Some complex modules may diff --git a/manual/install.texi b/manual/install.texi index 79ee45fcdb..663f8d59fb 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -186,8 +186,8 @@ new warnings cause the build with @option{-Werror} to fail), you can configure with @option{--disable-werror}. @item --disable-mathvec -By default for x86_64, @theglibc{} is built with vector math library. -Use this option to disable vector math library. +By default for x86_64, @theglibc{} is built with the vector math library. +Use this option to disable the vector math library. @item --build=@var{build-system} @itemx --host=@var{host-system} @@ -225,7 +225,7 @@ information for @theglibc{}. To build the library and related programs, type @code{make}. This will produce a lot of output, some of which may look like errors from -@code{make} but isn't. Look for error messages from @code{make} +@code{make} but aren't. Look for error messages from @code{make} containing @samp{***}. Those indicate that something is seriously wrong. The compilation process can take a long time, depending on the -- cgit v1.2.3 From 022dfdce000374b60aadfb0a5ed9a5c4c1dbd29b Mon Sep 17 00:00:00 2001 From: Stefan Liebler Date: Fri, 7 Oct 2016 09:56:46 +0200 Subject: Add configure check to test if gcc supports attribute ifunc. This patch adds a configure check to test if gcc supports attribute ifunc. The support can either be enabled in /gcc/config.gcc for one architecture in general by setting default_gnu_indirect_function variable to yes or by configuring gcc with --enable-gnu-indirect-function. The next patch rewrites libc_ifunc macro to use gcc attribute ifunc instead of inline assembly to generate the IFUNC symbols due to false debuginfo. If gcc does not support attribute ifunc, the old approach for generating ifunc'ed symbols is used. Then the debug-information is false. Thus it is recommended to use a gcc with indirect function support (See notes in INSTALL). After this patch-series these inline assemblies for ifunc-handling are not scattered in multiple files but are used only indirect via ifunc-macros and can simply removed in libc-symbols.h in future. If glibc is configured with --enable-multi-arch and gcc does not support attribute ifunc, a configure warning is dumped! ChangeLog: * config.h.in (HAVE_GCC_IFUNC): New undef. * configure.ac: Add check if gcc supports attribute ifunc feature. * configure: Regenerated. * manual/install.texi: Add recommendation for gcc with indirect-function support. * INSTALL: Regenerated. --- ChangeLog | 9 +++++++++ INSTALL | 9 +++++++++ NEWS | 9 +++++++++ config.h.in | 3 +++ configure | 42 ++++++++++++++++++++++++++++++++++++++++++ configure.ac | 33 +++++++++++++++++++++++++++++++++ manual/install.texi | 8 ++++++++ 7 files changed, 113 insertions(+) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 437667385a..bc0c75c3d1 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,12 @@ +2016-10-07 Stefan Liebler + + * config.h.in (HAVE_GCC_IFUNC): New undef. + * configure.ac: Add check if gcc supports attribute ifunc feature. + * configure: Regenerated. + * manual/install.texi: Add recommendation for gcc with + indirect-function support. + * INSTALL: Regenerated. + 2016-10-06 Joseph Myers * math/math.h [__GLIBC_USE (IEC_60559_BFP_EXT)] (iseqsig): New diff --git a/INSTALL b/INSTALL index 51f26bf962..b5acedcc96 100644 --- a/INSTALL +++ b/INSTALL @@ -359,6 +359,15 @@ build the GNU C Library: better code. As of release time, GCC 5.3 is the newest compiler verified to work to build the GNU C Library. + For multi-arch support it is recommended to use a GCC which has + been built with support for GNU indirect functions. This ensures + that correct debugging information is generated for functions + selected by IFUNC resolvers. This support can either be enabled by + configuring GCC with '--enable-gnu-indirect-function', or by + enabling it by default by setting 'default_gnu_indirect_function' + variable for a particular architecture in the GCC source file + 'gcc/config.gcc'. + You can use whatever compiler you like to compile programs that use the GNU C Library. diff --git a/NEWS b/NEWS index b5894afce3..b077d0a078 100644 --- a/NEWS +++ b/NEWS @@ -76,6 +76,15 @@ Version 2.25 The glibc stub resolver did not support these hooks, but the header file did not reflect that. +* For multi-arch support it is recommended to use a GCC which has + been built with support for GNU indirect functions. This ensures + that correct debugging information is generated for functions + selected by IFUNC resolvers. This support can either be enabled by + configuring GCC with '--enable-gnu-indirect-function', or by + enabling it by default by setting 'default_gnu_indirect_function' + variable for a particular architecture in the GCC source file + 'gcc/config.gcc'. + Security related changes: On ARM EABI (32-bit), generating a backtrace for execution contexts which diff --git a/config.h.in b/config.h.in index 8cd08b0e89..33757bd553 100644 --- a/config.h.in +++ b/config.h.in @@ -174,6 +174,9 @@ /* Define to 1 if STT_GNU_IFUNC support actually works. */ #define HAVE_IFUNC 0 +/* Define if gcc supports attribute ifunc. */ +#undef HAVE_GCC_IFUNC + /* Define if the linker defines __ehdr_start. */ #undef HAVE_EHDR_START diff --git a/configure b/configure index 6ff252744b..e80e0ad56c 100755 --- a/configure +++ b/configure @@ -3916,6 +3916,36 @@ fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_ld_gnu_indirect_function" >&5 $as_echo "$libc_cv_ld_gnu_indirect_function" >&6; } +# Check if gcc supports attribute ifunc as it is used in libc_ifunc macro. +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for gcc attribute ifunc support" >&5 +$as_echo_n "checking for gcc attribute ifunc support... " >&6; } +if ${libc_cv_gcc_indirect_function+:} false; then : + $as_echo_n "(cached) " >&6 +else + cat > conftest.c <&5 \ + 2>&5 ; then + if $READELF -s conftest.o | grep IFUNC >/dev/null 2>&5; then + libc_cv_gcc_indirect_function=yes + fi +fi +rm -f conftest* +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_gcc_indirect_function" >&5 +$as_echo "$libc_cv_gcc_indirect_function" >&6; } + if test x"$libc_cv_ld_gnu_indirect_function" != xyes; then if test x"$multi_arch" = xyes; then as_fn_error $? "--enable-multi-arch support requires assembler and linker support" "$LINENO" 5 @@ -3923,6 +3953,13 @@ if test x"$libc_cv_ld_gnu_indirect_function" != xyes; then multi_arch=no fi fi +if test x"$libc_cv_gcc_indirect_function" != xyes && + test x"$multi_arch" = xyes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: --enable-multi-arch support recommends a gcc with gnu-indirect-function support. +Please use a gcc which supports it by default or configure gcc with --enable-gnu-indirect-function" >&5 +$as_echo "$as_me: WARNING: --enable-multi-arch support recommends a gcc with gnu-indirect-function support. +Please use a gcc which supports it by default or configure gcc with --enable-gnu-indirect-function" >&2;} +fi multi_arch_d= if test x"$multi_arch" != xno; then multi_arch_d=/multiarch @@ -6518,6 +6555,11 @@ if test x"$libc_cv_ld_gnu_indirect_function" = xyes; then fi +if test x"$libc_cv_gcc_indirect_function" = xyes; then + $as_echo "#define HAVE_GCC_IFUNC 1" >>confdefs.h + +fi + # This is far from the AC_ARG_ENABLE that sets it so that a sysdeps # configure fragment can override the value to prevent this AC_DEFINE. diff --git a/configure.ac b/configure.ac index 9938ab0dc2..a64aeb9979 100644 --- a/configure.ac +++ b/configure.ac @@ -634,6 +634,30 @@ if ${CC-cc} $CFLAGS $CPPFLAGS $LDFLAGS \ fi rm -f conftest*]) +# Check if gcc supports attribute ifunc as it is used in libc_ifunc macro. +AC_CACHE_CHECK([for gcc attribute ifunc support], + libc_cv_gcc_indirect_function, [dnl +cat > conftest.c <&AS_MESSAGE_LOG_FD \ + 2>&AS_MESSAGE_LOG_FD ; then + if $READELF -s conftest.o | grep IFUNC >/dev/null 2>&AS_MESSAGE_LOG_FD; then + libc_cv_gcc_indirect_function=yes + fi +fi +rm -f conftest*]) + if test x"$libc_cv_ld_gnu_indirect_function" != xyes; then if test x"$multi_arch" = xyes; then AC_MSG_ERROR([--enable-multi-arch support requires assembler and linker support]) @@ -641,6 +665,11 @@ if test x"$libc_cv_ld_gnu_indirect_function" != xyes; then multi_arch=no fi fi +if test x"$libc_cv_gcc_indirect_function" != xyes && + test x"$multi_arch" = xyes; then + AC_MSG_WARN([--enable-multi-arch support recommends a gcc with gnu-indirect-function support. +Please use a gcc which supports it by default or configure gcc with --enable-gnu-indirect-function]) +fi multi_arch_d= if test x"$multi_arch" != xno; then multi_arch_d=/multiarch @@ -1782,6 +1811,10 @@ if test x"$libc_cv_ld_gnu_indirect_function" = xyes; then AC_DEFINE(HAVE_IFUNC) fi +if test x"$libc_cv_gcc_indirect_function" = xyes; then + AC_DEFINE(HAVE_GCC_IFUNC) +fi + # This is far from the AC_ARG_ENABLE that sets it so that a sysdeps # configure fragment can override the value to prevent this AC_DEFINE. AC_SUBST(use_nscd) diff --git a/manual/install.texi b/manual/install.texi index 663f8d59fb..de1c2030d3 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -402,6 +402,14 @@ the newest version of the compiler that is known to work for building release time, GCC 5.3 is the newest compiler verified to work to build @theglibc{}. +For multi-arch support it is recommended to use a GCC which has been built with +support for GNU indirect functions. This ensures that correct debugging +information is generated for functions selected by IFUNC resolvers. This +support can either be enabled by configuring GCC with +@samp{--enable-gnu-indirect-function}, or by enabling it by default by setting +@samp{default_gnu_indirect_function} variable for a particular architecture in +the GCC source file @file{gcc/config.gcc}. + You can use whatever compiler you like to compile programs that use @theglibc{}. -- cgit v1.2.3 From 23b5cae1af04f2d912910fdaf73cb482265798c1 Mon Sep 17 00:00:00 2001 From: Martin Galvan Date: Thu, 8 Dec 2016 18:59:02 +0530 Subject: Add pretty printers for the NPTL lock types This patch adds pretty printers for the following NPTL types: - pthread_mutex_t - pthread_mutexattr_t - pthread_cond_t - pthread_condattr_t - pthread_rwlock_t - pthread_rwlockattr_t To load the pretty printers into your gdb session, do the following: python import sys sys.path.insert(0, '/path/to/glibc/build/nptl/pretty-printers') end source /path/to/glibc/source/pretty-printers/nptl-printers.py You can check which printers are registered and enabled by issuing the 'info pretty-printer' gdb command. Printers should trigger automatically when trying to print a variable of one of the types mentioned above. The printers are architecture-independent, and were tested on an AMD64 running Ubuntu 14.04 and an x86 VM running Fedora 24. In order to work, the printers need to know the values of various flags that are scattered throughout pthread.h and pthreadP.h as enums and #defines. Since replicating these constants in the printers file itself would create a maintenance burden, I wrote a script called gen-py-const.awk that Makerules uses to extract the constants. This script is pretty much the same as gen-as-const.awk, except it doesn't cast the constant values to 'long' and is thorougly documented. The constants need only to be enumerated in a .pysym file, which is then referenced by a Make variable called gen-py-const-headers. As for the install directory, I discussed this with Mike Frysinger and Siddhesh Poyarekar, and we agreed that it can be handled in a separate patch, and shouldn't block merging of this one. In addition, I've written a series of test cases for the pretty printers. Each lock type (mutex, condvar and rwlock) has two test programs, one for itself and other for its related 'attributes' object. Each test program in turn has a PExpect-based Python script that drives gdb and compares its output to the expected printer's. The tests run on the glibc host, which is assumed to have both gdb and PExpect; if either is absent the tests will fail with code 77 (UNSUPPORTED). For cross-testing you should use cross-test-ssh.sh as test-wrapper. I've tested the printers on both native builds and a cross build using a Beaglebone Black running Debian, with the build system's filesystem shared with the board through NFS. Finally, I've written a README that explains all this and more. * INSTALL: Regenerated. * Makeconfig: Add comments and whitespace to make the control flow clearer. (+link-printers-tests, +link-pie-printers-tests, CFLAGS-printers-tests, installed-rtld-LDFLAGS, built-rtld-LDFLAGS, link-libc-rpath, link-libc-tests-after-rpath-link, link-libc-printers-tests): New. (rtld-LDFLAGS, rtld-tests-LDFLAGS, link-libc-tests-rpath-link, link-libc-tests): Use the new variables as required. * Makerules ($(py-const)): New rule. generated: Add $(py-const). * README.pretty-printers: New file. * Rules (tests-printers-programs, tests-printers-out, py-env): New. (others): Depend on $(py-const). (tests): Depend on $(tests-printers-programs) or $(tests-printers-out), as required. Pass $(tests-printers) to merge-test-results.sh. * manual/install.texi: Add requirements for testing the pretty printers. * nptl/Makefile (gen-py-const-headers, pretty-printers, tests-printers, CFLAGS-test-mutexattr-printers.c CFLAGS-test-mutex-printers.c, CFLAGS-test-condattr-printers.c, CFLAGS-test-cond-printers.c, CFLAGS-test-rwlockattr-printers.c CFLAGS-test-rwlock-printers.c, tests-printers-libs): Define. * nptl/nptl-printers.py: New file. * nptl/nptl_lock_constants.pysym: Likewise. * nptl/test-cond-printers.c: Likewise. * nptl/test-cond-printers.py: Likewise. * nptl/test-condattr-printers.c: Likewise. * nptl/test-condattr-printers.py: Likewise. * nptl/test-mutex-printers.c: Likewise. * nptl/test-mutex-printers.py: Likewise. * nptl/test-mutexattr-printers.c: Likewise. * nptl/test-mutexattr-printers.py: Likewise. * nptl/test-rwlock-printers.c: Likewise. * nptl/test-rwlock-printers.py: Likewise. * nptl/test-rwlockattr-printers.c: Likewise. * nptl/test-rwlockattr-printers.py: Likewise. * scripts/gen-py-const.awk: Likewise. * scripts/test_printers_common.py: Likewise. * scripts/test_printers_exceptions.py: Likewise. --- ChangeLog | 45 +++ INSTALL | 27 ++ Makeconfig | 76 ++++- Makerules | 46 +++ NEWS | 5 + README.pretty-printers | 169 ++++++++++ Rules | 44 ++- manual/install.texi | 30 ++ nptl/Makefile | 18 + nptl/nptl-printers.py | 633 ++++++++++++++++++++++++++++++++++++ nptl/nptl_lock_constants.pysym | 75 +++++ nptl/test-cond-printers.c | 57 ++++ nptl/test-cond-printers.py | 50 +++ nptl/test-condattr-printers.c | 94 ++++++ nptl/test-condattr-printers.py | 71 ++++ nptl/test-mutex-printers.c | 151 +++++++++ nptl/test-mutex-printers.py | 97 ++++++ nptl/test-mutexattr-printers.c | 144 ++++++++ nptl/test-mutexattr-printers.py | 101 ++++++ nptl/test-rwlock-printers.c | 78 +++++ nptl/test-rwlock-printers.py | 64 ++++ nptl/test-rwlockattr-printers.c | 98 ++++++ nptl/test-rwlockattr-printers.py | 73 +++++ scripts/gen-py-const.awk | 118 +++++++ scripts/test_printers_common.py | 364 +++++++++++++++++++++ scripts/test_printers_exceptions.py | 61 ++++ 26 files changed, 2769 insertions(+), 20 deletions(-) create mode 100644 README.pretty-printers create mode 100644 nptl/nptl-printers.py create mode 100644 nptl/nptl_lock_constants.pysym create mode 100644 nptl/test-cond-printers.c create mode 100644 nptl/test-cond-printers.py create mode 100644 nptl/test-condattr-printers.c create mode 100644 nptl/test-condattr-printers.py create mode 100644 nptl/test-mutex-printers.c create mode 100644 nptl/test-mutex-printers.py create mode 100644 nptl/test-mutexattr-printers.c create mode 100644 nptl/test-mutexattr-printers.py create mode 100644 nptl/test-rwlock-printers.c create mode 100644 nptl/test-rwlock-printers.py create mode 100644 nptl/test-rwlockattr-printers.c create mode 100644 nptl/test-rwlockattr-printers.py create mode 100644 scripts/gen-py-const.awk create mode 100644 scripts/test_printers_common.py create mode 100644 scripts/test_printers_exceptions.py (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 57ce74d446..d5ef99ce8d 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,48 @@ +2016-12-08 Martin Galvan + + * INSTALL: Regenerated. + * Makeconfig: Add comments and whitespace to make the control flow + clearer. + (+link-printers-tests, +link-pie-printers-tests, + CFLAGS-printers-tests, installed-rtld-LDFLAGS, + built-rtld-LDFLAGS, link-libc-rpath, + link-libc-tests-after-rpath-link, + link-libc-printers-tests): New. + (rtld-LDFLAGS, rtld-tests-LDFLAGS, link-libc-tests-rpath-link, + link-libc-tests): Use the new variables as required. + * Makerules ($(py-const)): New rule. + generated: Add $(py-const). + * README.pretty-printers: New file. + * Rules (tests-printers-programs, tests-printers-out, py-env): New. + (others): Depend on $(py-const). + (tests): Depend on $(tests-printers-programs) or + $(tests-printers-out), + as required. Pass $(tests-printers) to merge-test-results.sh. + * manual/install.texi: Add requirements for testing the pretty + printers. + * nptl/Makefile (gen-py-const-headers, pretty-printers, + tests-printers, CFLAGS-test-mutexattr-printers.c + CFLAGS-test-mutex-printers.c, CFLAGS-test-condattr-printers.c, + CFLAGS-test-cond-printers.c, CFLAGS-test-rwlockattr-printers.c + CFLAGS-test-rwlock-printers.c, tests-printers-libs): Define. + * nptl/nptl-printers.py: New file. + * nptl/nptl_lock_constants.pysym: Likewise. + * nptl/test-cond-printers.c: Likewise. + * nptl/test-cond-printers.py: Likewise. + * nptl/test-condattr-printers.c: Likewise. + * nptl/test-condattr-printers.py: Likewise. + * nptl/test-mutex-printers.c: Likewise. + * nptl/test-mutex-printers.py: Likewise. + * nptl/test-mutexattr-printers.c: Likewise. + * nptl/test-mutexattr-printers.py: Likewise. + * nptl/test-rwlock-printers.c: Likewise. + * nptl/test-rwlock-printers.py: Likewise. + * nptl/test-rwlockattr-printers.c: Likewise. + * nptl/test-rwlockattr-printers.py: Likewise. + * scripts/gen-py-const.awk: Likewise. + * scripts/test_printers_common.py: Likewise. + * scripts/test_printers_exceptions.py: Likewise. + 2016-12-07 Joseph Myers * scripts/build-many-glibcs.py (Context.__init__): Take strip diff --git a/INSTALL b/INSTALL index b5acedcc96..acb622a102 100644 --- a/INSTALL +++ b/INSTALL @@ -224,6 +224,33 @@ You can specify 'stop-on-test-failure=y' when running 'make check' to make the test run stop and exit with an error status immediately when a failure occurs. + The GNU C Library pretty printers come with their own set of scripts +for testing, which run together with the rest of the testsuite through +'make check'. These scripts require the following tools to run +successfully: + + * Python 2.7.6/3.4.3 or later + + Python is required for running the printers' test scripts. + + * PExpect 4.0 + + The printer tests drive GDB through test programs and compare its + output to the printers'. PExpect is used to capture the output of + GDB, and should be compatible with the Python version in your + system. + + * GDB 7.8 or later with support for Python 2.7.6/3.4.3 or later + + GDB itself needs to be configured with Python support in order to + use the pretty printers. Notice that your system having Python + available doesn't imply that GDB supports it, nor that your + system's Python and GDB's have the same version. + +If these tools are absent, the printer tests will report themselves as +'UNSUPPORTED'. Notice that some of the printer tests require the GNU C +Library to be compiled with debugging symbols. + To format the 'GNU C Library Reference Manual' for printing, type 'make dvi'. You need a working TeX installation to do this. The distribution builds the on-line formatted version of the manual, as Info diff --git a/Makeconfig b/Makeconfig index a7858607b0..e9d8da9d60 100644 --- a/Makeconfig +++ b/Makeconfig @@ -416,6 +416,11 @@ $(+link-pie-before-libc) $(rtld-tests-LDFLAGS) $(link-libc-tests) \ $(+link-pie-after-libc) $(call after-link,$@) endef +define +link-pie-printers-tests +$(+link-pie-before-libc) $(built-rtld-LDFLAGS) $(link-libc-printers-tests) \ + $(+link-pie-after-libc) +$(call after-link,$@) +endef endif # Command for statically linking programs with the C library. ifndef +link-static @@ -445,7 +450,8 @@ ifeq (yes,$(build-pie-default)) no-pie-ldflag = -no-pie +link = $(+link-pie) +link-tests = $(+link-pie-tests) -else ++link-printers-tests = $(+link-pie-printers-tests) +else # not build-pie-default +link-before-libc = $(CC) -nostdlib -nostartfiles -o $@ \ $(sysdep-LDFLAGS) $(LDFLAGS) $(LDFLAGS-$(@F)) \ $(combreloc-LDFLAGS) $(relro-LDFLAGS) $(hashstyle-LDFLAGS) \ @@ -466,51 +472,87 @@ $(+link-before-libc) $(rtld-tests-LDFLAGS) $(link-libc-tests) \ $(+link-after-libc) $(call after-link,$@) endef -endif -else +define +link-printers-tests +$(+link-before-libc) $(built-rtld-LDFLAGS) $(link-libc-printers-tests) \ + $(+link-after-libc) +$(call after-link,$@) +endef +endif # build-pie-default +else # build-static +link = $(+link-static) +link-tests = $(+link-static-tests) -endif -endif ++link-printers-tests = $(+link-static-tests) +endif # build-shared +endif # +link + +# The pretty printer test programs need to be compiled without optimizations +# so they won't confuse gdb. We could use either the 'GCC optimize' pragma +# or the 'optimize' function attribute to achieve this; however, at least on +# ARM, gcc always produces different debugging symbols when invoked with +# a -O greater than 0 than when invoked with -O0, regardless of anything else +# we're using to suppress optimizations. Therefore, we need to explicitly pass +# -O0 to it through CFLAGS. +# Additionally, the build system will try to -include $(common-objpfx)/config.h +# when compiling the tests, which will throw an error if some special macros +# (such as __OPTIMIZE__ and IS_IN_build) aren't defined. To avoid this, we +# tell gcc to define IS_IN_build. +CFLAGS-printers-tests := -O0 -ggdb3 -DIS_IN_build + ifeq (yes,$(build-shared)) +# These indicate whether to link using the built ld.so or the installed one. +installed-rtld-LDFLAGS = -Wl,-dynamic-linker=$(rtlddir)/$(rtld-installed-name) +built-rtld-LDFLAGS = -Wl,-dynamic-linker=$(elf-objpfx)ld.so + ifndef rtld-LDFLAGS -rtld-LDFLAGS = -Wl,-dynamic-linker=$(rtlddir)/$(rtld-installed-name) +rtld-LDFLAGS = $(installed-rtld-LDFLAGS) endif + ifndef rtld-tests-LDFLAGS ifeq (yes,$(build-hardcoded-path-in-tests)) -rtld-tests-LDFLAGS = -Wl,-dynamic-linker=$(elf-objpfx)ld.so +rtld-tests-LDFLAGS = $(built-rtld-LDFLAGS) else -rtld-tests-LDFLAGS = $(rtld-LDFLAGS) -endif -endif -endif +rtld-tests-LDFLAGS = $(installed-rtld-LDFLAGS) +endif # build-hardcoded-path-in-tests +endif # rtld-tests-LDFLAGS + +endif # build-shared + ifndef link-libc ifeq (yes,$(build-shared)) # We need the versioned name of libc.so in the deps of $(others) et al # so that the symlink to libc.so is created before anything tries to # run the linked programs. +link-libc-rpath = -Wl,-rpath=$(rpath-link) link-libc-rpath-link = -Wl,-rpath-link=$(rpath-link) + ifeq (yes,$(build-hardcoded-path-in-tests)) -link-libc-tests-rpath-link = -Wl,-rpath=$(rpath-link) +link-libc-tests-rpath-link = $(link-libc-rpath) else link-libc-tests-rpath-link = $(link-libc-rpath-link) -endif +endif # build-hardcoded-path-in-tests + link-libc-before-gnulib = $(common-objpfx)libc.so$(libc.so-version) \ $(common-objpfx)$(patsubst %,$(libtype.oS),c) \ $(as-needed) $(elf-objpfx)ld.so \ $(no-as-needed) link-libc = $(link-libc-rpath-link) $(link-libc-before-gnulib) $(gnulib) + +link-libc-tests-after-rpath-link = $(link-libc-before-gnulib) $(gnulib-tests) link-libc-tests = $(link-libc-tests-rpath-link) \ - $(link-libc-before-gnulib) $(gnulib-tests) + $(link-libc-tests-after-rpath-link) +# Pretty printer test programs always require rpath instead of rpath-link. +link-libc-printers-tests = $(link-libc-rpath) \ + $(link-libc-tests-after-rpath-link) + # This is how to find at build-time things that will be installed there. rpath-dirs = math elf dlfcn nss nis rt resolv crypt mathvec rpath-link = \ $(common-objdir):$(subst $(empty) ,:,$(patsubst ../$(subdir),.,$(rpath-dirs:%=$(common-objpfx)%))) -else +else # build-static link-libc = $(common-objpfx)libc.a $(otherlibs) $(gnulib) $(common-objpfx)libc.a $(gnulib) link-libc-tests = $(common-objpfx)libc.a $(otherlibs) $(gnulib-tests) $(common-objpfx)libc.a $(gnulib-tests) -endif -endif +endif # build-shared +endif # link-libc # Differences in the linkers on the various platforms. LDFLAGS-rpath-ORIGIN = -Wl,-rpath,'$$ORIGIN' diff --git a/Makerules b/Makerules index e865782b43..7214b2b43c 100644 --- a/Makerules +++ b/Makerules @@ -210,6 +210,52 @@ sed-remove-dotdot := -e 's@ *\([^ \/$$][^ \]*\)@ $$(..)\1@g' \ -e 's@^\([^ \/$$][^ \]*\)@$$(..)\1@g' endif +ifdef gen-py-const-headers +# We'll use a static pattern rule to match .pysym files with their +# corresponding generated .py files. +# The generated .py files go in the submodule's dir in the glibc build dir. +py-const-files := $(patsubst %.pysym,%.py,$(gen-py-const-headers)) +py-const-dir := $(objpfx) +py-const := $(addprefix $(py-const-dir),$(py-const-files)) +py-const-script := $(..)scripts/gen-py-const.awk + +# This is a hack we use to generate .py files with constants for Python +# pretty printers. It works the same way as gen-as-const. +# See scripts/gen-py-const.awk for details on how the awk | gcc mechanism +# works. +# +# $@.tmp and $@.tmp2 are temporary files we use to store the partial contents +# of the target file. We do this instead of just writing on $@ because, if the +# build process terminates prematurely, re-running Make wouldn't run this rule +# since Make would see that the target file already exists (despite it being +# incomplete). +# +# The sed line replaces "@name@SOME_NAME@value@SOME_VALUE@" strings from the +# output of 'gcc -S' with "SOME_NAME = SOME_VALUE" strings. +# The '-n' option, combined with the '/p' command, makes sed output only the +# modified lines instead of the whole input file. The output is redirected +# to a .py file; we'll import it in the pretty printers file to read +# the constants generated by gen-py-const.awk. +# The regex has two capturing groups, for SOME_NAME and SOME_VALUE +# respectively. Notice SOME_VALUE may be prepended by a special character, +# depending on the assembly syntax (e.g. immediates are prefixed by a '$' +# in AT&T x86, and by a '#' in ARM). We discard it using a complemented set +# before the second capturing group. +$(py-const): $(py-const-dir)%.py: %.pysym $(py-const-script) \ + $(common-before-compile) + $(make-target-directory) + $(AWK) -f $(py-const-script) $< \ + | $(CC) -S -o $@.tmp $(CFLAGS) $(CPPFLAGS) -x c - + echo '# GENERATED FILE\n' > $@.tmp2 + echo '# Constant definitions for pretty printers.' >> $@.tmp2 + echo '# See gen-py-const.awk for details.\n' >> $@.tmp2 + sed -n -r 's/^.*@name@([^@]+)@value@[^[:xdigit:]Xx-]*([[:xdigit:]Xx-]+)@.*/\1 = \2/p' \ + $@.tmp >> $@.tmp2 + mv -f $@.tmp2 $@ + rm -f $@.tmp + +generated += $(py-const) +endif # gen-py-const-headers ifdef gen-as-const-headers # Generating headers for assembly constants. diff --git a/NEWS b/NEWS index c2e973d183..922a5004ff 100644 --- a/NEWS +++ b/NEWS @@ -141,6 +141,11 @@ Version 2.25 variable for a particular architecture in the GCC source file 'gcc/config.gcc'. +* GDB pretty printers have been added for mutex and condition variable + structures in POSIX Threads. When installed and loaded in gdb these pretty + printers show various pthread variables in human-readable form when read + using the 'print' or 'display' commands in gdb. + Security related changes: On ARM EABI (32-bit), generating a backtrace for execution contexts which diff --git a/README.pretty-printers b/README.pretty-printers new file mode 100644 index 0000000000..86629001e6 --- /dev/null +++ b/README.pretty-printers @@ -0,0 +1,169 @@ +README for the glibc Python pretty printers +=========================================== + +Pretty printers are gdb extensions that allow it to print useful, human-readable +information about a program's variables. For example, for a pthread_mutex_t +gdb would usually output something like this: + +(gdb) print mutex +$1 = { + __data = { + __lock = 22020096, + __count = 0, + __owner = 0, + __nusers = 0, + __kind = 576, + __spins = 0, + __elision = 0, + __list = { + __prev = 0x0, + __next = 0x0 + } + }, + __size = "\000\000P\001", '\000' , "@\002", '\000' , + __align = 22020096 +} + +However, with a pretty printer gdb will output something like this: + +(gdb) print mutex +$1 = pthread_mutex_t = { + Type = Normal, + Status = Unlocked, + Robust = No, + Shared = No, + Protocol = Priority protect, + Priority ceiling = 42 +} + +Before printing a value, gdb will first check if there's a pretty printer +registered for it. If there is, it'll use it, otherwise it'll print the value +as usual. Pretty printers can be registered in various ways; for our purposes +we register them for the current objfile by calling +gdb.printing.register_pretty_printer(). + +Currently our printers are based on gdb.RegexpCollectionPrettyPrinter, which +means they'll be triggered if the type of the variable we're printing matches +a given regular expression. For example, MutexPrinter will be triggered if +our variable's type matches the regexp '^pthread_mutex_t$'. + +Besides the printers themselves, each module may have a constants file which the +printers will import. These constants are generated from C headers during the +build process, and need to be in the Python search path when loading the +printers. + + +Installing and loading +---------------------- + +The pretty printers and their constant files may be installed in different paths +for each distro, though gdb should be able to automatically load them by itself. +When in doubt, you can use the 'info pretty-printer' gdb command to list the +loaded pretty printers. + +If the printers aren't automatically loaded for some reason, you should add the +following to your .gdbinit: + +python +import sys +sys.path.insert(0, '/path/to/constants/file/directory') +end + +source /path/to/printers.py + +If you're building glibc manually, '/path/to/constants/file/directory' should be +'/path/to/glibc-build/submodule', where 'submodule' is e.g. nptl. + + +Testing +------- + +The pretty printers come with a small test suite based on PExpect, which is a +Python module with Expect-like features for spawning and controlling interactive +programs. Each printer has a corresponding C program and a Python script +that uses PExpect to drive gdb through the program and compare its output to +the expected printer's. + +The tests run on the glibc host, which is assumed to have both gdb and PExpect; +if any of those is absent the tests will fail with code 77 (UNSUPPORTED). +Native builds can be tested simply by doing 'make check'; cross builds must use +cross-test-ssh.sh as test-wrapper, like this: + +make test-wrapper='/path/to/scripts/cross-test-ssh.sh user@host' check + +(Remember to share the build system's filesystem with the glibc host's through +NFS or something similar). + +Running 'make check' on a cross build will only compile the test programs, +without running the scripts. + + +Adding new pretty printers +-------------------------- + +Adding new pretty printers to glibc requires following these steps: + +1. Identify which constants must be generated from C headers, and write the +corresponding .pysym file. See scripts/gen-py-const.awk for more information +on how this works. The name of the .pysym file must be added to the +'gen-py-const-headers' variable in your submodule's Makefile (without the .pysym +extension). + +2. Write the pretty printer code itself. For this you can follow the gdb +Python API documentation, and use the existing printers as examples. The printer +code must import the generated constants file (which will have the same name +as your .pysym file). The names of the pretty printer files must be added +to the 'pretty-printers' variable in your submodule's Makefile (without the .py +extension). + +3. Write the unit tests for your pretty printers. The build system calls each +test script passing it the paths to the test program source, the test program +binary, and the printer files you added to 'pretty-printers' in the previous +step. The test scripts, in turn, must import scripts/test_printers_common +and call the init_test function passing it, among other things, the name of the +set of pretty printers to enable (as seen by running 'info pretty-printer'). +You can use the existing unit tests as examples. + +4. Add the names of the pretty printer tests to the 'tests-printers' variable +in your submodule's Makefile (without extensions). In addition, for each test +program you must define a corresponding CFLAGS-* variable and set it to +$(CFLAGS-printers-tests) to ensure they're compiled correctly. For example, +test-foo-printer.c requires the following: + +CFLAGS-test-foo-printer.c := $(CFLAGS-printers-tests) + +Finally, if your programs need to be linked with a specific library, you can add +its name to the 'tests-printers-libs' variable in your submodule's Makefile. + + +Known issues +------------ + +* Pretty printers are inherently coupled to the code they're targetting, thus +any changes to the target code must also update the corresponding printers. +On the plus side, the printer code itself may serve as a kind of documentation +for the target code. + +* Older versions of the gdb Python API have a bug where +gdb.RegexpCollectionPrettyPrinter would not be able to get a value's real type +if it was typedef'd. This would cause gdb to ignore the pretty printers for +types like pthread_mutex_t, which is defined as: + +typedef union +{ + ... +} pthread_mutex_t; + +This was fixed in commit 1b588015839caafc608a6944a78aea170f5fb2f6, and released +as part of gdb 7.8. However, typedef'ing an already typedef'd type may cause +a similar issue, e.g.: + +typedef pthread_mutex_t mutex; +mutex a_mutex; + +Here, trying to print a_mutex won't trigger the pthread_mutex_t printer. + +* The test programs must be compiled without optimizations. This is necessary +because the test scripts rely on the C code structure being preserved when +stepping through the programs. Things like aggressive instruction reordering +or optimizing variables out may make this kind of testing impossible. diff --git a/Rules b/Rules index 466db076f0..714e917de4 100644 --- a/Rules +++ b/Rules @@ -108,17 +108,28 @@ endif .PHONY: others tests bench bench-build +# Test programs for the pretty printers. +tests-printers-programs := $(addprefix $(objpfx),$(tests-printers)) + +# .out files with the output of running the pretty printer tests. +tests-printers-out := $(patsubst %,$(objpfx)%.out,$(tests-printers)) + ifeq ($(build-programs),yes) others: $(addprefix $(objpfx),$(others) $(sysdep-others) $(extra-objs)) else others: $(addprefix $(objpfx),$(extra-objs)) endif + +# Generate constant files for Python pretty printers if required. +others: $(py-const) + ifeq ($(run-built-tests),no) tests: $(addprefix $(objpfx),$(filter-out $(tests-unsupported),$(tests)) \ - $(test-srcs)) $(tests-special) + $(test-srcs)) $(tests-special) \ + $(tests-printers-programs) xtests: tests $(xtests-special) else -tests: $(tests:%=$(objpfx)%.out) $(tests-special) +tests: $(tests:%=$(objpfx)%.out) $(tests-special) $(tests-printers-out) xtests: tests $(xtests:%=$(objpfx)%.out) $(xtests-special) endif @@ -131,7 +142,8 @@ tests-expected = $(tests) endif tests: $(..)scripts/merge-test-results.sh -s $(objpfx) $(subdir) \ - $(sort $(tests-expected) $(tests-special-notdir:.out=)) \ + $(sort $(tests-expected) $(tests-special-notdir:.out=) \ + $(tests-printers)) \ > $(objpfx)subdir-tests.sum xtests: $(..)scripts/merge-test-results.sh -s $(objpfx) $(subdir) \ @@ -241,6 +253,32 @@ endif endif # tests +ifneq "$(strip $(tests-printers))" "" +# We're defining this here for now; later it'll be defined at configure time +# inside Makeconfig. +PYTHON := python + +# Static pattern rule for building the test programs for the pretty printers. +$(tests-printers-programs): %: %.o $(tests-printers-libs) \ + $(sort $(filter $(common-objpfx)lib%,$(link-libc-static-tests))) \ + $(addprefix $(csu-objpfx),start.o) $(+preinit) $(+postinit) + $(+link-printers-tests) + +# Add the paths to the generated constants file and test_common_printers.py +# to PYTHONPATH so the test scripts can find them. +py-env := PYTHONPATH=$(py-const-dir):$(..)scripts:$${PYTHONPATH} + +# Static pattern rule that matches the test-* targets to their .c and .py +# prerequisites. It'll run the corresponding test script for each test program +# we compiled and place its output in the corresponding .out file. +# The pretty printer files and test_common_printers.py must be present for all. +$(tests-printers-out): $(objpfx)%.out: $(objpfx)% %.py %.c $(pretty-printers) \ + $(..)scripts/test_printers_common.py + $(test-wrapper-env) $(py-env) \ + $(PYTHON) $*.py $*.c $(objpfx)$* $(pretty-printers) > $@; \ + $(evaluate-test) +endif + .PHONY: distclean realclean subdir_distclean subdir_realclean \ subdir_clean subdir_mostlyclean subdir_testclean diff --git a/manual/install.texi b/manual/install.texi index de1c2030d3..35e02ada58 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -256,6 +256,36 @@ occurred. You can specify @samp{stop-on-test-failure=y} when running @code{make check} to make the test run stop and exit with an error status immediately when a failure occurs. +The @glibcadj{} pretty printers come with their own set of scripts for testing, +which run together with the rest of the testsuite through @code{make check}. +These scripts require the following tools to run successfully: + +@itemize @bullet +@item +Python 2.7.6/3.4.3 or later + +Python is required for running the printers' test scripts. + +@item PExpect 4.0 + +The printer tests drive GDB through test programs and compare its output +to the printers'. PExpect is used to capture the output of GDB, and should be +compatible with the Python version in your system. + +@item +GDB 7.8 or later with support for Python 2.7.6/3.4.3 or later + +GDB itself needs to be configured with Python support in order to use the +pretty printers. Notice that your system having Python available doesn't imply +that GDB supports it, nor that your system's Python and GDB's have the same +version. +@end itemize + +@noindent +If these tools are absent, the printer tests will report themselves as +@code{UNSUPPORTED}. Notice that some of the printer tests require @theglibc{} +to be compiled with debugging symbols. + To format the @cite{GNU C Library Reference Manual} for printing, type @w{@code{make dvi}}. You need a working @TeX{} installation to do this. The distribution builds the on-line formatted version of the diff --git a/nptl/Makefile b/nptl/Makefile index 11588fe996..7ac9196975 100644 --- a/nptl/Makefile +++ b/nptl/Makefile @@ -312,6 +312,24 @@ gen-as-const-headers = pthread-errnos.sym \ unwindbuf.sym \ lowlevelrobustlock.sym pthread-pi-defines.sym +gen-py-const-headers := nptl_lock_constants.pysym +pretty-printers := nptl-printers.py +tests-printers := test-mutexattr-printers test-mutex-printers \ + test-condattr-printers test-cond-printers \ + test-rwlockattr-printers test-rwlock-printers + +CFLAGS-test-mutexattr-printers.c := $(CFLAGS-printers-tests) +CFLAGS-test-mutex-printers.c := $(CFLAGS-printers-tests) +CFLAGS-test-condattr-printers.c := $(CFLAGS-printers-tests) +CFLAGS-test-cond-printers.c := $(CFLAGS-printers-tests) +CFLAGS-test-rwlockattr-printers.c := $(CFLAGS-printers-tests) +CFLAGS-test-rwlock-printers.c := $(CFLAGS-printers-tests) + +ifeq ($(build-shared),yes) +tests-printers-libs := $(shared-thread-library) +else +tests-printers-libs := $(static-thread-library) +endif LDFLAGS-pthread.so = -Wl,--enable-new-dtags,-z,nodelete,-z,initfirst diff --git a/nptl/nptl-printers.py b/nptl/nptl-printers.py new file mode 100644 index 0000000000..e402f232c7 --- /dev/null +++ b/nptl/nptl-printers.py @@ -0,0 +1,633 @@ +# Pretty printers for the NPTL lock types. +# +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +"""This file contains the gdb pretty printers for the following types: + + * pthread_mutex_t + * pthread_mutexattr_t + * pthread_cond_t + * pthread_condattr_t + * pthread_rwlock_t + * pthread_rwlockattr_t + +You can check which printers are registered and enabled by issuing the +'info pretty-printer' gdb command. Printers should trigger automatically when +trying to print a variable of one of the types mentioned above. +""" + +from __future__ import print_function + +import gdb +import gdb.printing +from nptl_lock_constants import * + +MUTEX_TYPES = { + PTHREAD_MUTEX_NORMAL: ('Type', 'Normal'), + PTHREAD_MUTEX_RECURSIVE: ('Type', 'Recursive'), + PTHREAD_MUTEX_ERRORCHECK: ('Type', 'Error check'), + PTHREAD_MUTEX_ADAPTIVE_NP: ('Type', 'Adaptive') +} + +class MutexPrinter(object): + """Pretty printer for pthread_mutex_t.""" + + def __init__(self, mutex): + """Initialize the printer's internal data structures. + + Args: + mutex: A gdb.value representing a pthread_mutex_t. + """ + + data = mutex['__data'] + self.lock = data['__lock'] + self.count = data['__count'] + self.owner = data['__owner'] + self.kind = data['__kind'] + self.values = [] + self.read_values() + + def to_string(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_mutex_t. + """ + + return 'pthread_mutex_t' + + def children(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_mutex_t. + """ + + return self.values + + def read_values(self): + """Read the mutex's info and store it in self.values. + + The data contained in self.values will be returned by the Iterator + created in self.children. + """ + + self.read_type() + self.read_status() + self.read_attributes() + self.read_misc_info() + + def read_type(self): + """Read the mutex's type.""" + + mutex_type = self.kind & PTHREAD_MUTEX_KIND_MASK + + # mutex_type must be casted to int because it's a gdb.Value + self.values.append(MUTEX_TYPES[int(mutex_type)]) + + def read_status(self): + """Read the mutex's status. + + For architectures which support lock elision, this method reads + whether the mutex appears as locked in memory (i.e. it may show it as + unlocked even after calling pthread_mutex_lock). + """ + + if self.kind == PTHREAD_MUTEX_DESTROYED: + self.values.append(('Status', 'Destroyed')) + elif self.kind & PTHREAD_MUTEX_ROBUST_NORMAL_NP: + self.read_status_robust() + else: + self.read_status_no_robust() + + def read_status_robust(self): + """Read the status of a robust mutex. + + In glibc robust mutexes are implemented in a very different way than + non-robust ones. This method reads their locking status, + whether it may have waiters, their registered owner (if any), + whether the owner is alive or not, and the status of the state + they're protecting. + """ + + if self.lock == PTHREAD_MUTEX_UNLOCKED: + self.values.append(('Status', 'Unlocked')) + else: + if self.lock & FUTEX_WAITERS: + self.values.append(('Status', 'Locked, possibly with waiters')) + else: + self.values.append(('Status', + 'Locked, possibly with no waiters')) + + if self.lock & FUTEX_OWNER_DIED: + self.values.append(('Owner ID', '%d (dead)' % self.owner)) + else: + self.values.append(('Owner ID', self.lock & FUTEX_TID_MASK)) + + if self.owner == PTHREAD_MUTEX_INCONSISTENT: + self.values.append(('State protected by this mutex', + 'Inconsistent')) + elif self.owner == PTHREAD_MUTEX_NOTRECOVERABLE: + self.values.append(('State protected by this mutex', + 'Not recoverable')) + + def read_status_no_robust(self): + """Read the status of a non-robust mutex. + + Read info on whether the mutex is locked, if it may have waiters + and its owner (if any). + """ + + lock_value = self.lock + + if self.kind & PTHREAD_MUTEX_PRIO_PROTECT_NP: + lock_value &= ~(PTHREAD_MUTEX_PRIO_CEILING_MASK) + + if lock_value == PTHREAD_MUTEX_UNLOCKED: + self.values.append(('Status', 'Unlocked')) + else: + if self.kind & PTHREAD_MUTEX_PRIO_INHERIT_NP: + waiters = self.lock & FUTEX_WAITERS + owner = self.lock & FUTEX_TID_MASK + else: + # Mutex protocol is PP or none + waiters = (self.lock != PTHREAD_MUTEX_LOCKED_NO_WAITERS) + owner = self.owner + + if waiters: + self.values.append(('Status', 'Locked, possibly with waiters')) + else: + self.values.append(('Status', + 'Locked, possibly with no waiters')) + + self.values.append(('Owner ID', owner)) + + def read_attributes(self): + """Read the mutex's attributes.""" + + if self.kind != PTHREAD_MUTEX_DESTROYED: + if self.kind & PTHREAD_MUTEX_ROBUST_NORMAL_NP: + self.values.append(('Robust', 'Yes')) + else: + self.values.append(('Robust', 'No')) + + # In glibc, robust mutexes always have their pshared flag set to + # 'shared' regardless of what the pshared flag of their + # mutexattr was. Therefore a robust mutex will act as shared + # even if it was initialized with a 'private' mutexattr. + if self.kind & PTHREAD_MUTEX_PSHARED_BIT: + self.values.append(('Shared', 'Yes')) + else: + self.values.append(('Shared', 'No')) + + if self.kind & PTHREAD_MUTEX_PRIO_INHERIT_NP: + self.values.append(('Protocol', 'Priority inherit')) + elif self.kind & PTHREAD_MUTEX_PRIO_PROTECT_NP: + prio_ceiling = ((self.lock & PTHREAD_MUTEX_PRIO_CEILING_MASK) + >> PTHREAD_MUTEX_PRIO_CEILING_SHIFT) + + self.values.append(('Protocol', 'Priority protect')) + self.values.append(('Priority ceiling', prio_ceiling)) + else: + # PTHREAD_PRIO_NONE + self.values.append(('Protocol', 'None')) + + def read_misc_info(self): + """Read miscellaneous info on the mutex. + + For now this reads the number of times a recursive mutex was locked + by the same thread. + """ + + mutex_type = self.kind & PTHREAD_MUTEX_KIND_MASK + + if mutex_type == PTHREAD_MUTEX_RECURSIVE and self.count > 1: + self.values.append(('Times locked recursively', self.count)) + +class MutexAttributesPrinter(object): + """Pretty printer for pthread_mutexattr_t. + + In the NPTL this is a type that's always casted to struct pthread_mutexattr + which has a single 'mutexkind' field containing the actual attributes. + """ + + def __init__(self, mutexattr): + """Initialize the printer's internal data structures. + + Args: + mutexattr: A gdb.value representing a pthread_mutexattr_t. + """ + + self.values = [] + + try: + mutexattr_struct = gdb.lookup_type('struct pthread_mutexattr') + self.mutexattr = mutexattr.cast(mutexattr_struct)['mutexkind'] + self.read_values() + except gdb.error: + # libpthread doesn't have debug symbols, thus we can't find the + # real struct type. Just print the union members. + self.values.append(('__size', mutexattr['__size'])) + self.values.append(('__align', mutexattr['__align'])) + + def to_string(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_mutexattr_t. + """ + + return 'pthread_mutexattr_t' + + def children(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_mutexattr_t. + """ + + return self.values + + def read_values(self): + """Read the mutexattr's info and store it in self.values. + + The data contained in self.values will be returned by the Iterator + created in self.children. + """ + + mutexattr_type = (self.mutexattr + & ~PTHREAD_MUTEXATTR_FLAG_BITS + & ~PTHREAD_MUTEX_NO_ELISION_NP) + + # mutexattr_type must be casted to int because it's a gdb.Value + self.values.append(MUTEX_TYPES[int(mutexattr_type)]) + + if self.mutexattr & PTHREAD_MUTEXATTR_FLAG_ROBUST: + self.values.append(('Robust', 'Yes')) + else: + self.values.append(('Robust', 'No')) + + if self.mutexattr & PTHREAD_MUTEXATTR_FLAG_PSHARED: + self.values.append(('Shared', 'Yes')) + else: + self.values.append(('Shared', 'No')) + + protocol = ((self.mutexattr & PTHREAD_MUTEXATTR_PROTOCOL_MASK) >> + PTHREAD_MUTEXATTR_PROTOCOL_SHIFT) + + if protocol == PTHREAD_PRIO_NONE: + self.values.append(('Protocol', 'None')) + elif protocol == PTHREAD_PRIO_INHERIT: + self.values.append(('Protocol', 'Priority inherit')) + elif protocol == PTHREAD_PRIO_PROTECT: + self.values.append(('Protocol', 'Priority protect')) + +CLOCK_IDS = { + CLOCK_REALTIME: 'CLOCK_REALTIME', + CLOCK_MONOTONIC: 'CLOCK_MONOTONIC', + CLOCK_PROCESS_CPUTIME_ID: 'CLOCK_PROCESS_CPUTIME_ID', + CLOCK_THREAD_CPUTIME_ID: 'CLOCK_THREAD_CPUTIME_ID', + CLOCK_MONOTONIC_RAW: 'CLOCK_MONOTONIC_RAW', + CLOCK_REALTIME_COARSE: 'CLOCK_REALTIME_COARSE', + CLOCK_MONOTONIC_COARSE: 'CLOCK_MONOTONIC_COARSE' +} + +class ConditionVariablePrinter(object): + """Pretty printer for pthread_cond_t.""" + + def __init__(self, cond): + """Initialize the printer's internal data structures. + + Args: + cond: A gdb.value representing a pthread_cond_t. + """ + + # Since PTHREAD_COND_SHARED is an integer, we need to cast it to void * + # to be able to compare it to the condvar's __data.__mutex member. + # + # While it looks like self.shared_value should be a class variable, + # that would result in it having an incorrect size if we're loading + # these printers through .gdbinit for a 64-bit objfile in AMD64. + # This is because gdb initially assumes the pointer size to be 4 bytes, + # and only sets it to 8 after loading the 64-bit objfiles. Since + # .gdbinit runs before any objfiles are loaded, this would effectively + # make self.shared_value have a size of 4, thus breaking later + # comparisons with pointers whose types are looked up at runtime. + void_ptr_type = gdb.lookup_type('void').pointer() + self.shared_value = gdb.Value(PTHREAD_COND_SHARED).cast(void_ptr_type) + + data = cond['__data'] + self.total_seq = data['__total_seq'] + self.mutex = data['__mutex'] + self.nwaiters = data['__nwaiters'] + self.values = [] + + self.read_values() + + def to_string(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_cond_t. + """ + + return 'pthread_cond_t' + + def children(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_cond_t. + """ + + return self.values + + def read_values(self): + """Read the condvar's info and store it in self.values. + + The data contained in self.values will be returned by the Iterator + created in self.children. + """ + + self.read_status() + self.read_attributes() + self.read_mutex_info() + + def read_status(self): + """Read the status of the condvar. + + This method reads whether the condvar is destroyed and how many threads + are waiting for it. + """ + + if self.total_seq == PTHREAD_COND_DESTROYED: + self.values.append(('Status', 'Destroyed')) + + self.values.append(('Threads waiting for this condvar', + self.nwaiters >> COND_NWAITERS_SHIFT)) + + def read_attributes(self): + """Read the condvar's attributes.""" + + clock_id = self.nwaiters & ((1 << COND_NWAITERS_SHIFT) - 1) + + # clock_id must be casted to int because it's a gdb.Value + self.values.append(('Clock ID', CLOCK_IDS[int(clock_id)])) + + shared = (self.mutex == self.shared_value) + + if shared: + self.values.append(('Shared', 'Yes')) + else: + self.values.append(('Shared', 'No')) + + def read_mutex_info(self): + """Read the data of the mutex this condvar is bound to. + + A pthread_cond_t's __data.__mutex member is a void * which + must be casted to pthread_mutex_t *. For shared condvars, this + member isn't recorded and has a special value instead. + """ + + if self.mutex and self.mutex != self.shared_value: + mutex_type = gdb.lookup_type('pthread_mutex_t') + mutex = self.mutex.cast(mutex_type.pointer()).dereference() + + self.values.append(('Mutex', mutex)) + +class ConditionVariableAttributesPrinter(object): + """Pretty printer for pthread_condattr_t. + + In the NPTL this is a type that's always casted to struct pthread_condattr, + which has a single 'value' field containing the actual attributes. + """ + + def __init__(self, condattr): + """Initialize the printer's internal data structures. + + Args: + condattr: A gdb.value representing a pthread_condattr_t. + """ + + self.values = [] + + try: + condattr_struct = gdb.lookup_type('struct pthread_condattr') + self.condattr = condattr.cast(condattr_struct)['value'] + self.read_values() + except gdb.error: + # libpthread doesn't have debug symbols, thus we can't find the + # real struct type. Just print the union members. + self.values.append(('__size', condattr['__size'])) + self.values.append(('__align', condattr['__align'])) + + def to_string(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_condattr_t. + """ + + return 'pthread_condattr_t' + + def children(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_condattr_t. + """ + + return self.values + + def read_values(self): + """Read the condattr's info and store it in self.values. + + The data contained in self.values will be returned by the Iterator + created in self.children. + """ + + clock_id = self.condattr & ((1 << COND_NWAITERS_SHIFT) - 1) + + # clock_id must be casted to int because it's a gdb.Value + self.values.append(('Clock ID', CLOCK_IDS[int(clock_id)])) + + if self.condattr & 1: + self.values.append(('Shared', 'Yes')) + else: + self.values.append(('Shared', 'No')) + +class RWLockPrinter(object): + """Pretty printer for pthread_rwlock_t.""" + + def __init__(self, rwlock): + """Initialize the printer's internal data structures. + + Args: + rwlock: A gdb.value representing a pthread_rwlock_t. + """ + + data = rwlock['__data'] + self.readers = data['__nr_readers'] + self.queued_readers = data['__nr_readers_queued'] + self.queued_writers = data['__nr_writers_queued'] + self.writer_id = data['__writer'] + self.shared = data['__shared'] + self.prefers_writers = data['__flags'] + self.values = [] + self.read_values() + + def to_string(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_rwlock_t. + """ + + return 'pthread_rwlock_t' + + def children(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_rwlock_t. + """ + + return self.values + + def read_values(self): + """Read the rwlock's info and store it in self.values. + + The data contained in self.values will be returned by the Iterator + created in self.children. + """ + + self.read_status() + self.read_attributes() + + def read_status(self): + """Read the status of the rwlock.""" + + # Right now pthread_rwlock_destroy doesn't do anything, so there's no + # way to check if an rwlock is destroyed. + + if self.writer_id: + self.values.append(('Status', 'Locked (Write)')) + self.values.append(('Writer ID', self.writer_id)) + elif self.readers: + self.values.append(('Status', 'Locked (Read)')) + self.values.append(('Readers', self.readers)) + else: + self.values.append(('Status', 'Unlocked')) + + self.values.append(('Queued readers', self.queued_readers)) + self.values.append(('Queued writers', self.queued_writers)) + + def read_attributes(self): + """Read the attributes of the rwlock.""" + + if self.shared: + self.values.append(('Shared', 'Yes')) + else: + self.values.append(('Shared', 'No')) + + if self.prefers_writers: + self.values.append(('Prefers', 'Writers')) + else: + self.values.append(('Prefers', 'Readers')) + +class RWLockAttributesPrinter(object): + """Pretty printer for pthread_rwlockattr_t. + + In the NPTL this is a type that's always casted to + struct pthread_rwlockattr, which has two fields ('lockkind' and 'pshared') + containing the actual attributes. + """ + + def __init__(self, rwlockattr): + """Initialize the printer's internal data structures. + + Args: + rwlockattr: A gdb.value representing a pthread_rwlockattr_t. + """ + + self.values = [] + + try: + rwlockattr_struct = gdb.lookup_type('struct pthread_rwlockattr') + self.rwlockattr = rwlockattr.cast(rwlockattr_struct) + self.read_values() + except gdb.error: + # libpthread doesn't have debug symbols, thus we can't find the + # real struct type. Just print the union members. + self.values.append(('__size', rwlockattr['__size'])) + self.values.append(('__align', rwlockattr['__align'])) + + def to_string(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_rwlockattr_t. + """ + + return 'pthread_rwlockattr_t' + + def children(self): + """gdb API function. + + This is called from gdb when we try to print a pthread_rwlockattr_t. + """ + + return self.values + + def read_values(self): + """Read the rwlockattr's info and store it in self.values. + + The data contained in self.values will be returned by the Iterator + created in self.children. + """ + + rwlock_type = self.rwlockattr['lockkind'] + shared = self.rwlockattr['pshared'] + + if shared == PTHREAD_PROCESS_SHARED: + self.values.append(('Shared', 'Yes')) + else: + # PTHREAD_PROCESS_PRIVATE + self.values.append(('Shared', 'No')) + + if (rwlock_type == PTHREAD_RWLOCK_PREFER_READER_NP or + rwlock_type == PTHREAD_RWLOCK_PREFER_WRITER_NP): + # This is a known bug. Using PTHREAD_RWLOCK_PREFER_WRITER_NP will + # still make the rwlock prefer readers. + self.values.append(('Prefers', 'Readers')) + elif rwlock_type == PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP: + self.values.append(('Prefers', 'Writers')) + +def register(objfile): + """Register the pretty printers within the given objfile.""" + + printer = gdb.printing.RegexpCollectionPrettyPrinter('glibc-pthread-locks') + + printer.add_printer('pthread_mutex_t', r'^pthread_mutex_t$', + MutexPrinter) + printer.add_printer('pthread_mutexattr_t', r'^pthread_mutexattr_t$', + MutexAttributesPrinter) + printer.add_printer('pthread_cond_t', r'^pthread_cond_t$', + ConditionVariablePrinter) + printer.add_printer('pthread_condattr_t', r'^pthread_condattr_t$', + ConditionVariableAttributesPrinter) + printer.add_printer('pthread_rwlock_t', r'^pthread_rwlock_t$', + RWLockPrinter) + printer.add_printer('pthread_rwlockattr_t', r'^pthread_rwlockattr_t$', + RWLockAttributesPrinter) + + if objfile == None: + objfile = gdb + + gdb.printing.register_pretty_printer(objfile, printer) + +register(gdb.current_objfile()) diff --git a/nptl/nptl_lock_constants.pysym b/nptl/nptl_lock_constants.pysym new file mode 100644 index 0000000000..303ec61213 --- /dev/null +++ b/nptl/nptl_lock_constants.pysym @@ -0,0 +1,75 @@ +#include + +-- Mutex types +PTHREAD_MUTEX_KIND_MASK PTHREAD_MUTEX_KIND_MASK_NP +PTHREAD_MUTEX_NORMAL +PTHREAD_MUTEX_RECURSIVE PTHREAD_MUTEX_RECURSIVE_NP +PTHREAD_MUTEX_ERRORCHECK PTHREAD_MUTEX_ERRORCHECK_NP +PTHREAD_MUTEX_ADAPTIVE_NP + +-- Mutex status +-- These are hardcoded all over the code; there are no enums/macros for them. +PTHREAD_MUTEX_DESTROYED -1 +PTHREAD_MUTEX_UNLOCKED 0 +PTHREAD_MUTEX_LOCKED_NO_WAITERS 1 + +-- For robust mutexes +PTHREAD_MUTEX_INCONSISTENT +PTHREAD_MUTEX_NOTRECOVERABLE +FUTEX_OWNER_DIED + +-- For robust and PI mutexes +FUTEX_WAITERS +FUTEX_TID_MASK + +-- Mutex attributes +PTHREAD_MUTEX_ROBUST_NORMAL_NP +PTHREAD_MUTEX_PRIO_INHERIT_NP +PTHREAD_MUTEX_PRIO_PROTECT_NP +PTHREAD_MUTEX_PSHARED_BIT +PTHREAD_MUTEX_PRIO_CEILING_SHIFT +PTHREAD_MUTEX_PRIO_CEILING_MASK + +-- Mutex attribute flags +PTHREAD_MUTEXATTR_PROTOCOL_SHIFT +PTHREAD_MUTEXATTR_PROTOCOL_MASK +PTHREAD_MUTEXATTR_PRIO_CEILING_MASK +PTHREAD_MUTEXATTR_FLAG_ROBUST +PTHREAD_MUTEXATTR_FLAG_PSHARED +PTHREAD_MUTEXATTR_FLAG_BITS +PTHREAD_MUTEX_NO_ELISION_NP + +-- Priority protocols +PTHREAD_PRIO_NONE +PTHREAD_PRIO_INHERIT +PTHREAD_PRIO_PROTECT + +-- These values are hardcoded as well: +-- Value of __mutex for shared condvars. +PTHREAD_COND_SHARED (void *)~0l + +-- Value of __total_seq for destroyed condvars. +PTHREAD_COND_DESTROYED -1ull + +-- __nwaiters encodes the number of threads waiting on a condvar +-- and the clock ID. +-- __nwaiters >> COND_NWAITERS_SHIFT gives us the number of waiters. +COND_NWAITERS_SHIFT + +-- Condvar clock IDs +CLOCK_REALTIME +CLOCK_MONOTONIC +CLOCK_PROCESS_CPUTIME_ID +CLOCK_THREAD_CPUTIME_ID +CLOCK_MONOTONIC_RAW +CLOCK_REALTIME_COARSE +CLOCK_MONOTONIC_COARSE + +-- Rwlock attributes +PTHREAD_RWLOCK_PREFER_READER_NP +PTHREAD_RWLOCK_PREFER_WRITER_NP +PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP + +-- 'Shared' attribute values +PTHREAD_PROCESS_PRIVATE +PTHREAD_PROCESS_SHARED diff --git a/nptl/test-cond-printers.c b/nptl/test-cond-printers.c new file mode 100644 index 0000000000..0f2a5f4827 --- /dev/null +++ b/nptl/test-cond-printers.c @@ -0,0 +1,57 @@ +/* Helper program for testing the pthread_cond_t pretty printer. + + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Keep the calls to the pthread_* functions on separate lines to make it easy + to advance through the program using the gdb 'next' command. */ + +#include +#include + +#define PASS 0 +#define FAIL 1 + +static int test_status_destroyed (pthread_cond_t *condvar); + +int +main (void) +{ + pthread_cond_t condvar; + pthread_condattr_t attr; + int result = FAIL; + + if (pthread_condattr_init (&attr) == 0 + && test_status_destroyed (&condvar) == PASS) + result = PASS; + /* Else, one of the pthread_cond* functions failed. */ + + return result; +} + +/* Initializes CONDVAR, then destroys it. */ +static int +test_status_destroyed (pthread_cond_t *condvar) +{ + int result = FAIL; + + if (pthread_cond_init (condvar, NULL) == 0 + && pthread_cond_destroy (condvar) == 0) + result = PASS; /* Test status (destroyed). */ + + return result; +} diff --git a/nptl/test-cond-printers.py b/nptl/test-cond-printers.py new file mode 100644 index 0000000000..af0e12eb97 --- /dev/null +++ b/nptl/test-cond-printers.py @@ -0,0 +1,50 @@ +# Common tests for the ConditionVariablePrinter class. +# +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +import sys + +from test_printers_common import * + +test_source = sys.argv[1] +test_bin = sys.argv[2] +printer_files = sys.argv[3:] +printer_names = ['global glibc-pthread-locks'] + +try: + init_test(test_bin, printer_files, printer_names) + go_to_main() + + var = 'condvar' + to_string = 'pthread_cond_t' + + break_at(test_source, 'Test status (destroyed)') + continue_cmd() # Go to test_status_destroyed + test_printer(var, to_string, {'Status': 'Destroyed'}) + + continue_cmd() # Exit + +except (NoLineError, pexpect.TIMEOUT) as exception: + print('Error: {0}'.format(exception)) + result = FAIL + +else: + print('Test succeeded.') + result = PASS + +exit(result) diff --git a/nptl/test-condattr-printers.c b/nptl/test-condattr-printers.c new file mode 100644 index 0000000000..4db4098827 --- /dev/null +++ b/nptl/test-condattr-printers.c @@ -0,0 +1,94 @@ +/* Helper program for testing the pthread_cond_t and pthread_condattr_t + pretty printers. + + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Keep the calls to the pthread_* functions on separate lines to make it easy + to advance through the program using the gdb 'next' command. */ + +#include +#include + +#define PASS 0 +#define FAIL 1 + +static int condvar_reinit (pthread_cond_t *condvar, + const pthread_condattr_t *attr); +static int test_setclock (pthread_cond_t *condvar, pthread_condattr_t *attr); +static int test_setpshared (pthread_cond_t *condvar, pthread_condattr_t *attr); + +/* Need these so we don't have lines longer than 79 chars. */ +#define SET_SHARED(attr, shared) pthread_condattr_setpshared (attr, shared) + +int +main (void) +{ + pthread_cond_t condvar; + pthread_condattr_t attr; + int result = FAIL; + + if (pthread_condattr_init (&attr) == 0 + && pthread_cond_init (&condvar, NULL) == 0 + && test_setclock (&condvar, &attr) == PASS + && test_setpshared (&condvar, &attr) == PASS) + result = PASS; + /* Else, one of the pthread_cond* functions failed. */ + + return result; +} + +/* Destroys CONDVAR and re-initializes it using ATTR. */ +static int +condvar_reinit (pthread_cond_t *condvar, const pthread_condattr_t *attr) +{ + int result = FAIL; + + if (pthread_cond_destroy (condvar) == 0 + && pthread_cond_init (condvar, attr) == 0) + result = PASS; + + return result; +} + +/* Tests setting the clock ID attribute. */ +static int +test_setclock (pthread_cond_t *condvar, pthread_condattr_t *attr) +{ + int result = FAIL; + + if (pthread_condattr_setclock (attr, CLOCK_REALTIME) == 0 /* Set clock. */ + && condvar_reinit (condvar, attr) == PASS) + result = PASS; + + return result; +} + +/* Tests setting whether the condvar can be shared between processes. */ +static int +test_setpshared (pthread_cond_t *condvar, pthread_condattr_t *attr) +{ + int result = FAIL; + + if (SET_SHARED (attr, PTHREAD_PROCESS_SHARED) == 0 /* Set shared. */ + && condvar_reinit (condvar, attr) == PASS + && SET_SHARED (attr, PTHREAD_PROCESS_PRIVATE) == 0 + && condvar_reinit (condvar, attr) == PASS) + result = PASS; + + return result; +} diff --git a/nptl/test-condattr-printers.py b/nptl/test-condattr-printers.py new file mode 100644 index 0000000000..7ea01db158 --- /dev/null +++ b/nptl/test-condattr-printers.py @@ -0,0 +1,71 @@ +# Common tests for the ConditionVariablePrinter and +# ConditionVariableAttributesPrinter classes. +# +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +import sys + +from test_printers_common import * + +test_source = sys.argv[1] +test_bin = sys.argv[2] +printer_files = sys.argv[3:] +printer_names = ['global glibc-pthread-locks'] + +try: + init_test(test_bin, printer_files, printer_names) + go_to_main() + + check_debug_symbol('struct pthread_condattr') + + condvar_var = 'condvar' + condvar_to_string = 'pthread_cond_t' + + attr_var = 'attr' + attr_to_string = 'pthread_condattr_t' + + break_at(test_source, 'Set clock') + continue_cmd() # Go to test_setclock + next_cmd(2) + test_printer(condvar_var, condvar_to_string, {'Clock ID': 'CLOCK_REALTIME'}) + test_printer(attr_var, attr_to_string, {'Clock ID': 'CLOCK_REALTIME'}) + + break_at(test_source, 'Set shared') + continue_cmd() # Go to test_setpshared + next_cmd(2) + test_printer(condvar_var, condvar_to_string, {'Shared': 'Yes'}) + test_printer(attr_var, attr_to_string, {'Shared': 'Yes'}) + next_cmd(2) + test_printer(condvar_var, condvar_to_string, {'Shared': 'No'}) + test_printer(attr_var, attr_to_string, {'Shared': 'No'}) + + continue_cmd() # Exit + +except (NoLineError, pexpect.TIMEOUT) as exception: + print('Error: {0}'.format(exception)) + result = FAIL + +except DebugError as exception: + print(exception) + result = UNSUPPORTED + +else: + print('Test succeeded.') + result = PASS + +exit(result) diff --git a/nptl/test-mutex-printers.c b/nptl/test-mutex-printers.c new file mode 100644 index 0000000000..b973e82840 --- /dev/null +++ b/nptl/test-mutex-printers.c @@ -0,0 +1,151 @@ +/* Helper program for testing the pthread_mutex_t pretty printer. + + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Keep the calls to the pthread_* functions on separate lines to make it easy + to advance through the program using the gdb 'next' command. */ + +#include +#include +#include + +#define PASS 0 +#define FAIL 1 + +static int test_status_destroyed (pthread_mutex_t *mutex); +static int test_status_no_robust (pthread_mutex_t *mutex, + pthread_mutexattr_t *attr); +static int test_status_robust (pthread_mutex_t *mutex, + pthread_mutexattr_t *attr); +static int test_locking_state_robust (pthread_mutex_t *mutex); +static void *thread_func (void *arg); +static int test_recursive_locks (pthread_mutex_t *mutex, + pthread_mutexattr_t *attr); + +int +main (void) +{ + pthread_mutex_t mutex; + pthread_mutexattr_t attr; + int result = FAIL; + + if (pthread_mutexattr_init (&attr) == 0 + && test_status_destroyed (&mutex) == PASS + && test_status_no_robust (&mutex, &attr) == PASS + && test_status_robust (&mutex, &attr) == PASS + && test_recursive_locks (&mutex, &attr) == PASS) + result = PASS; + /* Else, one of the pthread_mutex* functions failed. */ + + return result; +} + +/* Initializes MUTEX, then destroys it. */ +static int +test_status_destroyed (pthread_mutex_t *mutex) +{ + int result = FAIL; + + if (pthread_mutex_init (mutex, NULL) == 0 + && pthread_mutex_destroy (mutex) == 0) + result = PASS; /* Test status (destroyed). */ + + return result; +} + +/* Tests locking of non-robust mutexes. */ +static int +test_status_no_robust (pthread_mutex_t *mutex, pthread_mutexattr_t *attr) +{ + int result = FAIL; + + if (pthread_mutexattr_setrobust (attr, PTHREAD_MUTEX_STALLED) == 0 + && pthread_mutex_init (mutex, attr) == 0 + && pthread_mutex_lock (mutex) == 0 /* Test status (non-robust). */ + && pthread_mutex_unlock (mutex) == 0 + && pthread_mutex_destroy (mutex) == 0) + result = PASS; + + return result; +} + +/* Tests locking of robust mutexes. */ +static int +test_status_robust (pthread_mutex_t *mutex, pthread_mutexattr_t *attr) +{ + int result = FAIL; + + if (pthread_mutexattr_setrobust (attr, PTHREAD_MUTEX_ROBUST) == 0 + && pthread_mutex_init (mutex, attr) == 0 + && test_locking_state_robust (mutex) == PASS /* Test status (robust). */ + && pthread_mutex_destroy (mutex) == 0) + result = PASS; + + return result; +} + +/* Tests locking and state corruption of robust mutexes. We'll mark it as + inconsistent, then not recoverable. */ +static int +test_locking_state_robust (pthread_mutex_t *mutex) +{ + int result = FAIL; + pthread_t thread; + + if (pthread_create (&thread, NULL, thread_func, mutex) == 0 /* Create. */ + && pthread_join (thread, NULL) == 0 + && pthread_mutex_lock (mutex) == EOWNERDEAD /* Test locking (robust). */ + && pthread_mutex_unlock (mutex) == 0) + result = PASS; + + return result; +} + +/* Function to be called by the child thread when testing robust mutexes. */ +static void * +thread_func (void *arg) +{ + pthread_mutex_t *mutex = (pthread_mutex_t *)arg; + + if (pthread_mutex_lock (mutex) != 0) /* Thread function. */ + exit (FAIL); + + /* Thread terminates without unlocking the mutex, thus marking it as + inconsistent. */ + return NULL; +} + +/* Tests locking the mutex multiple times in a row. */ +static int +test_recursive_locks (pthread_mutex_t *mutex, pthread_mutexattr_t *attr) +{ + int result = FAIL; + + if (pthread_mutexattr_settype (attr, PTHREAD_MUTEX_RECURSIVE) == 0 + && pthread_mutex_init (mutex, attr) == 0 + && pthread_mutex_lock (mutex) == 0 + && pthread_mutex_lock (mutex) == 0 + && pthread_mutex_lock (mutex) == 0 /* Test recursive locks. */ + && pthread_mutex_unlock (mutex) == 0 + && pthread_mutex_unlock (mutex) == 0 + && pthread_mutex_unlock (mutex) == 0 + && pthread_mutex_destroy (mutex) == 0) + result = PASS; + + return result; +} diff --git a/nptl/test-mutex-printers.py b/nptl/test-mutex-printers.py new file mode 100644 index 0000000000..7f542adcd7 --- /dev/null +++ b/nptl/test-mutex-printers.py @@ -0,0 +1,97 @@ +# Tests for the MutexPrinter class. +# +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +import sys + +from test_printers_common import * + +test_source = sys.argv[1] +test_bin = sys.argv[2] +printer_files = sys.argv[3:] +printer_names = ['global glibc-pthread-locks'] + +try: + init_test(test_bin, printer_files, printer_names) + go_to_main() + + var = 'mutex' + to_string = 'pthread_mutex_t' + + break_at(test_source, 'Test status (destroyed)') + continue_cmd() # Go to test_status_destroyed + test_printer(var, to_string, {'Status': 'Destroyed'}) + + break_at(test_source, 'Test status (non-robust)') + continue_cmd() # Go to test_status_no_robust + test_printer(var, to_string, {'Status': 'Unlocked'}) + next_cmd() + thread_id = get_current_thread_lwpid() + test_printer(var, to_string, {'Status': 'Locked, possibly with no waiters', + 'Owner ID': thread_id}) + + break_at(test_source, 'Test status (robust)') + continue_cmd() # Go to test_status_robust + test_printer(var, to_string, {'Status': 'Unlocked'}) + + # We'll now test the robust mutex locking states. We'll create a new + # thread that will lock a robust mutex and exit without unlocking it. + break_at(test_source, 'Create') + continue_cmd() # Go to test_locking_state_robust + # Set a breakpoint for the new thread to hit. + break_at(test_source, 'Thread function') + continue_cmd() + # By now the new thread is created and has hit its breakpoint. + set_scheduler_locking(True) + parent = 1 + child = 2 + select_thread(child) + child_id = get_current_thread_lwpid() + # We've got the new thread's ID. + select_thread(parent) + # Make the new thread finish its function while we wait. + continue_cmd(thread=child) + # The new thread should be dead by now. + break_at(test_source, 'Test locking (robust)') + continue_cmd() + test_printer(var, to_string, {'Owner ID': r'{0} \(dead\)'.format(child_id)}) + # Try to lock and unlock the mutex. + next_cmd() + test_printer(var, to_string, {'Owner ID': thread_id, + 'State protected by this mutex': 'Inconsistent'}) + next_cmd() + test_printer(var, to_string, {'Status': 'Unlocked', + 'State protected by this mutex': 'Not recoverable'}) + set_scheduler_locking(False) + + break_at(test_source, 'Test recursive locks') + continue_cmd() # Go to test_recursive_locks + test_printer(var, to_string, {'Times locked recursively': '2'}) + next_cmd() + test_printer(var, to_string, {'Times locked recursively': '3'}) + continue_cmd() # Exit + +except (NoLineError, pexpect.TIMEOUT) as exception: + print('Error: {0}'.format(exception)) + result = FAIL + +else: + print('Test succeeded.') + result = PASS + +exit(result) diff --git a/nptl/test-mutexattr-printers.c b/nptl/test-mutexattr-printers.c new file mode 100644 index 0000000000..9ecfff76c3 --- /dev/null +++ b/nptl/test-mutexattr-printers.c @@ -0,0 +1,144 @@ +/* Helper program for testing the pthread_mutex_t and pthread_mutexattr_t + pretty printers. + + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Keep the calls to the pthread_* functions on separate lines to make it easy + to advance through the program using the gdb 'next' command. */ + +#include + +#define PASS 0 +#define FAIL 1 +#define PRIOCEILING 42 + +/* Need these so we don't have lines longer than 79 chars. */ +#define SET_TYPE(attr, type) pthread_mutexattr_settype (attr, type) +#define SET_ROBUST(attr, robust) pthread_mutexattr_setrobust (attr, robust) +#define SET_SHARED(attr, shared) pthread_mutexattr_setpshared (attr, shared) +#define SET_PROTOCOL(attr, protocol) \ + pthread_mutexattr_setprotocol (attr, protocol) +#define SET_PRIOCEILING(mutex, prioceiling, old_ceiling) \ + pthread_mutex_setprioceiling (mutex, prioceiling, old_ceiling) + +static int mutex_reinit (pthread_mutex_t *mutex, + const pthread_mutexattr_t *attr); +static int test_settype (pthread_mutex_t *mutex, pthread_mutexattr_t *attr); +static int test_setrobust (pthread_mutex_t *mutex, pthread_mutexattr_t *attr); +static int test_setpshared (pthread_mutex_t *mutex, pthread_mutexattr_t *attr); +static int test_setprotocol (pthread_mutex_t *mutex, + pthread_mutexattr_t *attr); + +int +main (void) +{ + pthread_mutex_t mutex; + pthread_mutexattr_t attr; + int result = FAIL; + + if (pthread_mutexattr_init (&attr) == 0 + && pthread_mutex_init (&mutex, NULL) == 0 + && test_settype (&mutex, &attr) == PASS + && test_setrobust (&mutex, &attr) == PASS + && test_setpshared (&mutex, &attr) == PASS + && test_setprotocol (&mutex, &attr) == PASS) + result = PASS; + /* Else, one of the pthread_mutex* functions failed. */ + + return result; +} + +/* Destroys MUTEX and re-initializes it using ATTR. */ +static int +mutex_reinit (pthread_mutex_t *mutex, const pthread_mutexattr_t *attr) +{ + int result = FAIL; + + if (pthread_mutex_destroy (mutex) == 0 + && pthread_mutex_init (mutex, attr) == 0) + result = PASS; + + return result; +} + +/* Tests setting the mutex type. */ +static int +test_settype (pthread_mutex_t *mutex, pthread_mutexattr_t *attr) +{ + int result = FAIL; + + if (SET_TYPE (attr, PTHREAD_MUTEX_ERRORCHECK) == 0 /* Set type. */ + && mutex_reinit (mutex, attr) == 0 + && SET_TYPE (attr, PTHREAD_MUTEX_RECURSIVE) == 0 + && mutex_reinit (mutex, attr) == 0 + && SET_TYPE (attr, PTHREAD_MUTEX_NORMAL) == 0 + && mutex_reinit (mutex, attr) == 0) + result = PASS; + + return result; +} + +/* Tests setting whether the mutex is robust. */ +static int +test_setrobust (pthread_mutex_t *mutex, pthread_mutexattr_t *attr) +{ + int result = FAIL; + + if (SET_ROBUST (attr, PTHREAD_MUTEX_ROBUST) == 0 /* Set robust. */ + && mutex_reinit (mutex, attr) == 0 + && SET_ROBUST (attr, PTHREAD_MUTEX_STALLED) == 0 + && mutex_reinit (mutex, attr) == 0) + result = PASS; + + return result; +} + +/* Tests setting whether the mutex can be shared between processes. */ +static int +test_setpshared (pthread_mutex_t *mutex, pthread_mutexattr_t *attr) +{ + int result = FAIL; + + if (SET_SHARED (attr, PTHREAD_PROCESS_SHARED) == 0 /* Set shared. */ + && mutex_reinit (mutex, attr) == 0 + && SET_SHARED (attr, PTHREAD_PROCESS_PRIVATE) == 0 + && mutex_reinit (mutex, attr) == 0) + result = PASS; + + return result; +} + +/* Tests setting the mutex protocol and, for Priority Protect, the Priority + Ceiling. */ +static int +test_setprotocol (pthread_mutex_t *mutex, pthread_mutexattr_t *attr) +{ + int result = FAIL; + int old_prioceiling; + + if (SET_PROTOCOL (attr, PTHREAD_PRIO_INHERIT) == 0 /* Set protocol. */ + && mutex_reinit (mutex, attr) == 0 + && SET_PROTOCOL (attr, PTHREAD_PRIO_PROTECT) == 0 + && mutex_reinit (mutex, attr) == 0 + && SET_PRIOCEILING(mutex, PRIOCEILING, &old_prioceiling) == 0 + && SET_PROTOCOL (attr, PTHREAD_PRIO_NONE) == 0 + && mutex_reinit (mutex, attr) == 0) + result = PASS; + + return result; +} diff --git a/nptl/test-mutexattr-printers.py b/nptl/test-mutexattr-printers.py new file mode 100644 index 0000000000..4464723fbe --- /dev/null +++ b/nptl/test-mutexattr-printers.py @@ -0,0 +1,101 @@ +# Common tests for the MutexPrinter and MutexAttributesPrinter classes. +# +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +import sys + +from test_printers_common import * + +test_source = sys.argv[1] +test_bin = sys.argv[2] +printer_files = sys.argv[3:] +printer_names = ['global glibc-pthread-locks'] +PRIOCEILING = 42 + +try: + init_test(test_bin, printer_files, printer_names) + go_to_main() + + check_debug_symbol('struct pthread_mutexattr') + + mutex_var = 'mutex' + mutex_to_string = 'pthread_mutex_t' + + attr_var = 'attr' + attr_to_string = 'pthread_mutexattr_t' + + break_at(test_source, 'Set type') + continue_cmd() # Go to test_settype + next_cmd(2) + test_printer(attr_var, attr_to_string, {'Type': 'Error check'}) + test_printer(mutex_var, mutex_to_string, {'Type': 'Error check'}) + next_cmd(2) + test_printer(attr_var, attr_to_string, {'Type': 'Recursive'}) + test_printer(mutex_var, mutex_to_string, {'Type': 'Recursive'}) + next_cmd(2) + test_printer(attr_var, attr_to_string, {'Type': 'Normal'}) + test_printer(mutex_var, mutex_to_string, {'Type': 'Normal'}) + + break_at(test_source, 'Set robust') + continue_cmd() # Go to test_setrobust + next_cmd(2) + test_printer(attr_var, attr_to_string, {'Robust': 'Yes'}) + test_printer(mutex_var, mutex_to_string, {'Robust': 'Yes'}) + next_cmd(2) + test_printer(attr_var, attr_to_string, {'Robust': 'No'}) + test_printer(mutex_var, mutex_to_string, {'Robust': 'No'}) + + break_at(test_source, 'Set shared') + continue_cmd() # Go to test_setpshared + next_cmd(2) + test_printer(attr_var, attr_to_string, {'Shared': 'Yes'}) + test_printer(mutex_var, mutex_to_string, {'Shared': 'Yes'}) + next_cmd(2) + test_printer(attr_var, attr_to_string, {'Shared': 'No'}) + test_printer(mutex_var, mutex_to_string, {'Shared': 'No'}) + + break_at(test_source, 'Set protocol') + continue_cmd() # Go to test_setprotocol + next_cmd(2) + test_printer(attr_var, attr_to_string, {'Protocol': 'Priority inherit'}) + test_printer(mutex_var, mutex_to_string, {'Protocol': 'Priority inherit'}) + next_cmd(2) + test_printer(attr_var, attr_to_string, {'Protocol': 'Priority protect'}) + test_printer(mutex_var, mutex_to_string, {'Protocol': 'Priority protect'}) + next_cmd(2) + test_printer(mutex_var, mutex_to_string, {'Priority ceiling': + str(PRIOCEILING)}) + next_cmd() + test_printer(attr_var, attr_to_string, {'Protocol': 'None'}) + test_printer(mutex_var, mutex_to_string, {'Protocol': 'None'}) + + continue_cmd() # Exit + +except (NoLineError, pexpect.TIMEOUT) as exception: + print('Error: {0}'.format(exception)) + result = FAIL + +except DebugError as exception: + print(exception) + result = UNSUPPORTED + +else: + print('Test succeeded.') + result = PASS + +exit(result) diff --git a/nptl/test-rwlock-printers.c b/nptl/test-rwlock-printers.c new file mode 100644 index 0000000000..dbbe9b80d6 --- /dev/null +++ b/nptl/test-rwlock-printers.c @@ -0,0 +1,78 @@ +/* Helper program for testing the pthread_rwlock_t pretty printer. + + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Keep the calls to the pthread_* functions on separate lines to make it easy + to advance through the program using the gdb 'next' command. */ + +#include + +#define PASS 0 +#define FAIL 1 + +static int test_locking_reader (pthread_rwlock_t *rwlock); +static int test_locking_writer (pthread_rwlock_t *rwlock); + +int +main (void) +{ + pthread_rwlock_t rwlock; + + int result = FAIL; + + if (test_locking_reader (&rwlock) == PASS + && test_locking_writer (&rwlock) == PASS) + result = PASS; + /* Else, one of the pthread_rwlock* functions failed. */ + + return result; +} + +/* Tests locking the rwlock multiple times as a reader. */ +static int +test_locking_reader (pthread_rwlock_t *rwlock) +{ + int result = FAIL; + + if (pthread_rwlock_init (rwlock, NULL) == 0 + && pthread_rwlock_rdlock (rwlock) == 0 /* Test locking (reader). */ + && pthread_rwlock_rdlock (rwlock) == 0 + && pthread_rwlock_rdlock (rwlock) == 0 + && pthread_rwlock_unlock (rwlock) == 0 + && pthread_rwlock_unlock (rwlock) == 0 + && pthread_rwlock_unlock (rwlock) == 0 + && pthread_rwlock_destroy (rwlock) == 0) + result = PASS; + + return result; +} + +/* Tests locking the rwlock as a writer. */ +static int +test_locking_writer (pthread_rwlock_t *rwlock) +{ + int result = FAIL; + + if (pthread_rwlock_init (rwlock, NULL) == 0 + && pthread_rwlock_wrlock (rwlock) == 0 /* Test locking (writer). */ + && pthread_rwlock_unlock (rwlock) == 0 + && pthread_rwlock_destroy (rwlock) == 0) + result = PASS; + + return result; +} diff --git a/nptl/test-rwlock-printers.py b/nptl/test-rwlock-printers.py new file mode 100644 index 0000000000..b972fa60d0 --- /dev/null +++ b/nptl/test-rwlock-printers.py @@ -0,0 +1,64 @@ +# Common tests for the RWLockPrinter class. +# +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +import sys + +from test_printers_common import * + +test_source = sys.argv[1] +test_bin = sys.argv[2] +printer_files = sys.argv[3:] +printer_names = ['global glibc-pthread-locks'] + +try: + init_test(test_bin, printer_files, printer_names) + go_to_main() + + var = 'rwlock' + to_string = 'pthread_rwlock_t' + + break_at(test_source, 'Test locking (reader)') + continue_cmd() # Go to test_locking_reader + test_printer(var, to_string, {'Status': 'Unlocked'}) + next_cmd() + test_printer(var, to_string, {'Status': r'Locked \(Read\)', 'Readers': '1'}) + next_cmd() + test_printer(var, to_string, {'Readers': '2'}) + next_cmd() + test_printer(var, to_string, {'Readers': '3'}) + + break_at(test_source, 'Test locking (writer)') + continue_cmd() # Go to test_locking_writer + test_printer(var, to_string, {'Status': 'Unlocked'}) + next_cmd() + thread_id = get_current_thread_lwpid() + test_printer(var, to_string, {'Status': r'Locked \(Write\)', + 'Writer ID': thread_id}) + + continue_cmd() # Exit + +except (NoLineError, pexpect.TIMEOUT) as exception: + print('Error: {0}'.format(exception)) + result = FAIL + +else: + print('Test succeeded.') + result = PASS + +exit(result) diff --git a/nptl/test-rwlockattr-printers.c b/nptl/test-rwlockattr-printers.c new file mode 100644 index 0000000000..d12facf41c --- /dev/null +++ b/nptl/test-rwlockattr-printers.c @@ -0,0 +1,98 @@ +/* Helper program for testing the pthread_rwlock_t and pthread_rwlockattr_t + pretty printers. + + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Keep the calls to the pthread_* functions on separate lines to make it easy + to advance through the program using the gdb 'next' command. */ + +#include + +#define PASS 0 +#define FAIL 1 + +/* Need these so we don't have lines longer than 79 chars. */ +#define SET_KIND(attr, kind) pthread_rwlockattr_setkind_np (attr, kind) +#define SET_SHARED(attr, shared) pthread_rwlockattr_setpshared (attr, shared) + +static int rwlock_reinit (pthread_rwlock_t *rwlock, + const pthread_rwlockattr_t *attr); +static int test_setkind_np (pthread_rwlock_t *rwlock, + pthread_rwlockattr_t *attr); +static int test_setpshared (pthread_rwlock_t *rwlock, + pthread_rwlockattr_t *attr); + +int +main (void) +{ + pthread_rwlock_t rwlock; + pthread_rwlockattr_t attr; + int result = FAIL; + + if (pthread_rwlockattr_init (&attr) == 0 + && pthread_rwlock_init (&rwlock, NULL) == 0 + && test_setkind_np (&rwlock, &attr) == PASS + && test_setpshared (&rwlock, &attr) == PASS) + result = PASS; + /* Else, one of the pthread_rwlock* functions failed. */ + + return result; +} + +/* Destroys RWLOCK and re-initializes it using ATTR. */ +static int +rwlock_reinit (pthread_rwlock_t *rwlock, const pthread_rwlockattr_t *attr) +{ + int result = FAIL; + + if (pthread_rwlock_destroy (rwlock) == 0 + && pthread_rwlock_init (rwlock, attr) == 0) + result = PASS; + + return result; +} + +/* Tests setting whether the rwlock prefers readers or writers. */ +static int +test_setkind_np (pthread_rwlock_t *rwlock, pthread_rwlockattr_t *attr) +{ + int result = FAIL; + + if (SET_KIND (attr, PTHREAD_RWLOCK_PREFER_READER_NP) == 0 /* Set kind. */ + && rwlock_reinit (rwlock, attr) == PASS + && SET_KIND (attr, PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP) == 0 + && rwlock_reinit (rwlock, attr) == PASS) + result = PASS; + + return result; +} + +/* Tests setting whether the rwlock can be shared between processes. */ +static int +test_setpshared (pthread_rwlock_t *rwlock, pthread_rwlockattr_t *attr) +{ + int result = FAIL; + + if (SET_SHARED (attr, PTHREAD_PROCESS_SHARED) == 0 /* Set shared. */ + && rwlock_reinit (rwlock, attr) == PASS + && SET_SHARED (attr, PTHREAD_PROCESS_PRIVATE) == 0 + && rwlock_reinit (rwlock, attr) == PASS) + result = PASS; + + return result; +} diff --git a/nptl/test-rwlockattr-printers.py b/nptl/test-rwlockattr-printers.py new file mode 100644 index 0000000000..1ca2dc6c33 --- /dev/null +++ b/nptl/test-rwlockattr-printers.py @@ -0,0 +1,73 @@ +# Common tests for the RWLockPrinter and RWLockAttributesPrinter classes. +# +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +import sys + +from test_printers_common import * + +test_source = sys.argv[1] +test_bin = sys.argv[2] +printer_files = sys.argv[3:] +printer_names = ['global glibc-pthread-locks'] + +try: + init_test(test_bin, printer_files, printer_names) + go_to_main() + + check_debug_symbol('struct pthread_rwlockattr') + + rwlock_var = 'rwlock' + rwlock_to_string = 'pthread_rwlock_t' + + attr_var = 'attr' + attr_to_string = 'pthread_rwlockattr_t' + + break_at(test_source, 'Set kind') + continue_cmd() # Go to test_setkind_np + next_cmd(2) + test_printer(rwlock_var, rwlock_to_string, {'Prefers': 'Readers'}) + test_printer(attr_var, attr_to_string, {'Prefers': 'Readers'}) + next_cmd(2) + test_printer(rwlock_var, rwlock_to_string, {'Prefers': 'Writers'}) + test_printer(attr_var, attr_to_string, {'Prefers': 'Writers'}) + + break_at(test_source, 'Set shared') + continue_cmd() # Go to test_setpshared + next_cmd(2) + test_printer(rwlock_var, rwlock_to_string, {'Shared': 'Yes'}) + test_printer(attr_var, attr_to_string, {'Shared': 'Yes'}) + next_cmd(2) + test_printer(rwlock_var, rwlock_to_string, {'Shared': 'No'}) + test_printer(attr_var, attr_to_string, {'Shared': 'No'}) + + continue_cmd() # Exit + +except (NoLineError, pexpect.TIMEOUT) as exception: + print('Error: {0}'.format(exception)) + result = FAIL + +except DebugError as exception: + print(exception) + result = UNSUPPORTED + +else: + print('Test succeeded.') + result = PASS + +exit(result) diff --git a/scripts/gen-py-const.awk b/scripts/gen-py-const.awk new file mode 100644 index 0000000000..4586f59d89 --- /dev/null +++ b/scripts/gen-py-const.awk @@ -0,0 +1,118 @@ +# Script to generate constants for Python pretty printers. +# +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +# This script is a smaller version of the clever gen-asm-const.awk hack used to +# generate ASM constants from .sym files. We'll use this to generate constants +# for Python pretty printers. +# +# The input to this script are .pysym files that look like: +# #C_Preprocessor_Directive... +# NAME1 +# NAME2 expression... +# +# A line giving just a name implies an expression consisting of just that name. +# Comments start with '--'. +# +# The output of this script is a 'dummy' function containing 'asm' declarations +# for each non-preprocessor line in the .pysym file. The expression values +# will appear as input operands to the 'asm' declaration. For example, if we +# have: +# +# /* header.h */ +# #define MACRO 42 +# +# struct S { +# char c1; +# char c2; +# char c3; +# }; +# +# enum E { +# ZERO, +# ONE +# }; +# +# /* symbols.pysym */ +# #include +# #include "header.h" +# -- This is a comment +# MACRO +# C3_OFFSET offsetof(struct S, c3) +# E_ONE ONE +# +# the output will be: +# +# #include +# #include "header.h" +# void dummy(void) +# { +# asm ("@name@MACRO@value@%0@" : : "i" (MACRO)); +# asm ("@name@C3_OFFSET@value@%0@" : : "i" (offsetof(struct S, c3))); +# asm ("@name@E_ONE@value@%0@" : : "i" (ONE)); +# } +# +# We'll later feed this output to gcc -S. Since '-S' tells gcc to compile but +# not assemble, gcc will output something like: +# +# dummy: +# ... +# @name@MACRO@value@$42@ +# @name@C3_OFFSET@value@$2@ +# @name@E_ONE@value@$1@ +# +# Finally, we can process that output to extract the constant values. +# Notice gcc may prepend a special character such as '$' to each value. + +# found_symbol indicates whether we found a non-comment, non-preprocessor line. +BEGIN { found_symbol = 0 } + +# C preprocessor directives go straight through. +/^#/ { print; next; } + +# Skip comments. +/--/ { next; } + +# Trim leading whitespace. +{ sub(/^[[:blank:]]*/, ""); } + +# If we found a non-comment, non-preprocessor line, print the 'dummy' function +# header. +NF > 0 && !found_symbol { + print "void dummy(void)\n{"; + found_symbol = 1; +} + +# If the line contains just a name, duplicate it so we can use that name +# as the value of the expression. +NF == 1 { sub(/^.*$/, "& &"); } + +# If a line contains a name and an expression... +NF > 1 { + name = $1; + + # Remove any characters before the second field. + sub(/^[^[:blank:]]+[[:blank:]]+/, ""); + + # '$0' ends up being everything that appeared after the first field + # separator. + printf " asm (\"@name@%s@value@%0@\" : : \"i\" (%s));\n", name, $0; +} + +# Close the 'dummy' function. +END { if (found_symbol) print "}"; } diff --git a/scripts/test_printers_common.py b/scripts/test_printers_common.py new file mode 100644 index 0000000000..c79d7e3be2 --- /dev/null +++ b/scripts/test_printers_common.py @@ -0,0 +1,364 @@ +# Common functions and variables for testing the Python pretty printers. +# +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +"""These tests require PExpect 4.0 or newer. + +Exported constants: + PASS, FAIL, UNSUPPORTED (int): Test exit codes, as per evaluate-test.sh. +""" + +import os +import re +from test_printers_exceptions import * + +PASS = 0 +FAIL = 1 +UNSUPPORTED = 77 + +gdb_bin = 'gdb' +gdb_options = '-q -nx' +gdb_invocation = '{0} {1}'.format(gdb_bin, gdb_options) +pexpect_min_version = 4 +gdb_min_version = (7, 8) +encoding = 'utf-8' + +try: + import pexpect +except ImportError: + print('PExpect 4.0 or newer must be installed to test the pretty printers.') + exit(UNSUPPORTED) + +pexpect_version = pexpect.__version__.split('.')[0] + +if int(pexpect_version) < pexpect_min_version: + print('PExpect 4.0 or newer must be installed to test the pretty printers.') + exit(UNSUPPORTED) + +if not pexpect.which(gdb_bin): + print('gdb 7.8 or newer must be installed to test the pretty printers.') + exit(UNSUPPORTED) + +timeout = 5 +TIMEOUTFACTOR = os.environ.get('TIMEOUTFACTOR') + +if TIMEOUTFACTOR: + timeout = int(TIMEOUTFACTOR) + +try: + # Check the gdb version. + version_cmd = '{0} --version'.format(gdb_invocation, timeout=timeout) + gdb_version_out = pexpect.run(version_cmd, encoding=encoding) + + # The gdb version string is "GNU gdb ", where + # PKGVERSION can be any text. We assume that there'll always be a space + # between PKGVERSION and the version number for the sake of the regexp. + version_match = re.search(r'GNU gdb .* ([1-9]+)\.([0-9]+)', gdb_version_out) + + if not version_match: + print('The gdb version string (gdb -v) is incorrectly formatted.') + exit(UNSUPPORTED) + + gdb_version = (int(version_match.group(1)), int(version_match.group(2))) + + if gdb_version < gdb_min_version: + print('gdb 7.8 or newer must be installed to test the pretty printers.') + exit(UNSUPPORTED) + + # Check if gdb supports Python. + gdb_python_cmd = '{0} -ex "python import os" -batch'.format(gdb_invocation, + timeout=timeout) + gdb_python_error = pexpect.run(gdb_python_cmd, encoding=encoding) + + if gdb_python_error: + print('gdb must have python support to test the pretty printers.') + exit(UNSUPPORTED) + + # If everything's ok, spawn the gdb process we'll use for testing. + gdb = pexpect.spawn(gdb_invocation, echo=False, timeout=timeout, + encoding=encoding) + gdb_prompt = u'\(gdb\)' + gdb.expect(gdb_prompt) + +except pexpect.ExceptionPexpect as exception: + print('Error: {0}'.format(exception)) + exit(FAIL) + +def test(command, pattern=None): + """Sends 'command' to gdb and expects the given 'pattern'. + + If 'pattern' is None, simply consumes everything up to and including + the gdb prompt. + + Args: + command (string): The command we'll send to gdb. + pattern (raw string): A pattern the gdb output should match. + + Returns: + string: The string that matched 'pattern', or an empty string if + 'pattern' was None. + """ + + match = '' + + gdb.sendline(command) + + if pattern: + # PExpect does a non-greedy match for '+' and '*'. Since it can't look + # ahead on the gdb output stream, if 'pattern' ends with a '+' or a '*' + # we may end up matching only part of the required output. + # To avoid this, we'll consume 'pattern' and anything that follows it + # up to and including the gdb prompt, then extract 'pattern' later. + index = gdb.expect([u'{0}.+{1}'.format(pattern, gdb_prompt), + pexpect.TIMEOUT]) + + if index == 0: + # gdb.after now contains the whole match. Extract the text that + # matches 'pattern'. + match = re.match(pattern, gdb.after, re.DOTALL).group() + elif index == 1: + # We got a timeout exception. Print information on what caused it + # and bail out. + error = ('Response does not match the expected pattern.\n' + 'Command: {0}\n' + 'Expected pattern: {1}\n' + 'Response: {2}'.format(command, pattern, gdb.before)) + + raise pexpect.TIMEOUT(error) + else: + # Consume just the the gdb prompt. + gdb.expect(gdb_prompt) + + return match + +def init_test(test_bin, printer_files, printer_names): + """Loads the test binary file and the required pretty printers to gdb. + + Args: + test_bin (string): The name of the test binary file. + pretty_printers (list of strings): A list with the names of the pretty + printer files. + """ + + # Load all the pretty printer files. We're assuming these are safe. + for printer_file in printer_files: + test('source {0}'.format(printer_file)) + + # Disable all the pretty printers. + test('disable pretty-printer', r'0 of [0-9]+ printers enabled') + + # Enable only the required printers. + for printer in printer_names: + test('enable pretty-printer {0}'.format(printer), + r'[1-9][0-9]* of [1-9]+ printers enabled') + + # Finally, load the test binary. + test('file {0}'.format(test_bin)) + +def go_to_main(): + """Executes a gdb 'start' command, which takes us to main.""" + + test('start', r'main') + +def get_line_number(file_name, string): + """Returns the number of the line in which 'string' appears within a file. + + Args: + file_name (string): The name of the file we'll search through. + string (string): The string we'll look for. + + Returns: + int: The number of the line in which 'string' appears, starting from 1. + """ + number = -1 + + with open(file_name) as src_file: + for i, line in enumerate(src_file): + if string in line: + number = i + 1 + break + + if number == -1: + raise NoLineError(file_name, string) + + return number + +def break_at(file_name, string, temporary=True, thread=None): + """Places a breakpoint on the first line in 'file_name' containing 'string'. + + 'string' is usually a comment like "Stop here". Notice this may fail unless + the comment is placed inline next to actual code, e.g.: + + ... + /* Stop here */ + ... + + may fail, while: + + ... + some_func(); /* Stop here */ + ... + + will succeed. + + If 'thread' isn't None, the breakpoint will be set for all the threads. + Otherwise, it'll be set only for 'thread'. + + Args: + file_name (string): The name of the file we'll place the breakpoint in. + string (string): A string we'll look for inside the file. + We'll place a breakpoint on the line which contains it. + temporary (bool): Whether the breakpoint should be automatically deleted + after we reach it. + thread (int): The number of the thread we'll place the breakpoint for, + as seen by gdb. If specified, it should be greater than zero. + """ + + if not thread: + thread_str = '' + else: + thread_str = 'thread {0}'.format(thread) + + if temporary: + command = 'tbreak' + break_type = 'Temporary breakpoint' + else: + command = 'break' + break_type = 'Breakpoint' + + line_number = str(get_line_number(file_name, string)) + + test('{0} {1}:{2} {3}'.format(command, file_name, line_number, thread_str), + r'{0} [0-9]+ at 0x[a-f0-9]+: file {1}, line {2}\.'.format(break_type, + file_name, + line_number)) + +def continue_cmd(thread=None): + """Executes a gdb 'continue' command. + + If 'thread' isn't None, the command will be applied to all the threads. + Otherwise, it'll be applied only to 'thread'. + + Args: + thread (int): The number of the thread we'll apply the command to, + as seen by gdb. If specified, it should be greater than zero. + """ + + if not thread: + command = 'continue' + else: + command = 'thread apply {0} continue'.format(thread) + + test(command) + +def next_cmd(count=1, thread=None): + """Executes a gdb 'next' command. + + If 'thread' isn't None, the command will be applied to all the threads. + Otherwise, it'll be applied only to 'thread'. + + Args: + count (int): The 'count' argument of the 'next' command. + thread (int): The number of the thread we'll apply the command to, + as seen by gdb. If specified, it should be greater than zero. + """ + + if not thread: + command = 'next' + else: + command = 'thread apply {0} next' + + test('{0} {1}'.format(command, count)) + +def select_thread(thread): + """Selects the thread indicated by 'thread'. + + Args: + thread (int): The number of the thread we'll switch to, as seen by gdb. + This should be greater than zero. + """ + + if thread > 0: + test('thread {0}'.format(thread)) + +def get_current_thread_lwpid(): + """Gets the current thread's Lightweight Process ID. + + Returns: + string: The current thread's LWP ID. + """ + + # It's easier to get the LWP ID through the Python API than the gdb CLI. + command = 'python print(gdb.selected_thread().ptid[1])' + + return test(command, r'[0-9]+') + +def set_scheduler_locking(mode): + """Executes the gdb 'set scheduler-locking' command. + + Args: + mode (bool): Whether the scheduler locking mode should be 'on'. + """ + modes = { + True: 'on', + False: 'off' + } + + test('set scheduler-locking {0}'.format(modes[mode])) + +def test_printer(var, to_string, children=None, is_ptr=True): + """ Tests the output of a pretty printer. + + For a variable called 'var', this tests whether its associated printer + outputs the expected 'to_string' and children (if any). + + Args: + var (string): The name of the variable we'll print. + to_string (raw string): The expected output of the printer's 'to_string' + method. + children (map {raw string->raw string}): A map with the expected output + of the printer's children' method. + is_ptr (bool): Whether 'var' is a pointer, and thus should be + dereferenced. + """ + + if is_ptr: + var = '*{0}'.format(var) + + test('print {0}'.format(var), to_string) + + if children: + for name, value in children.items(): + # Children are shown as 'name = value'. + test('print {0}'.format(var), r'{0} = {1}'.format(name, value)) + +def check_debug_symbol(symbol): + """ Tests whether a given debugging symbol exists. + + If the symbol doesn't exist, raises a DebugError. + + Args: + symbol (string): The symbol we're going to check for. + """ + + try: + test('ptype {0}'.format(symbol), r'type = {0}'.format(symbol)) + + except pexpect.TIMEOUT: + # The symbol doesn't exist. + raise DebugError(symbol) diff --git a/scripts/test_printers_exceptions.py b/scripts/test_printers_exceptions.py new file mode 100644 index 0000000000..17034b544e --- /dev/null +++ b/scripts/test_printers_exceptions.py @@ -0,0 +1,61 @@ +# Exception classes used when testing the Python pretty printers. +# +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +class NoLineError(Exception): + """Custom exception to indicate that a test file doesn't contain + the requested string. + """ + + def __init__(self, file_name, string): + """Constructor. + + Args: + file_name (string): The name of the test file. + string (string): The string that was requested. + """ + + super(NoLineError, self).__init__() + self.file_name = file_name + self.string = string + + def __str__(self): + """Shows a readable representation of the exception.""" + + return ('File {0} has no line containing the following string: {1}' + .format(self.file_name, self.string)) + +class DebugError(Exception): + """Custom exception to indicate that a required debugging symbol is missing. + """ + + def __init__(self, symbol): + """Constructor. + + Args: + symbol (string): The name of the entity whose debug info is missing. + """ + + super(DebugError, self).__init__() + self.symbol = symbol + + def __str__(self): + """Shows a readable representation of the exception.""" + + return ('The required debugging information for {0} is missing.' + .format(self.symbol)) -- cgit v1.2.3 From 03baef1c9cfb396d76cae20a00aee657871e79c4 Mon Sep 17 00:00:00 2001 From: Nick Alcock Date: Mon, 26 Dec 2016 10:08:18 +0100 Subject: Configure support for --enable-stack-protector [BZ #7065] This adds =all and =strong, with obvious semantics, defaulting to off. We don't validate the value of the option yet: that's in a later patch. Nor do we use it for anything at this stage. We differentiate between 'the compiler understands -fstack-protector' and 'the user wanted -fstack-protector' so that we can pass -fno-stack-protector in appropriate places even if the user didn't want to turn on -fstack-protector for other parts. (This helps us overcome another existing limitation, that glibc doesn't work with GCCs hacked to pass in -fstack-protector by default.) We also arrange to set the STACK_PROTECTOR_LEVEL #define to a value appropriate for the stack-protection level in use for each file in particular. --- ChangeLog | 13 +++++ INSTALL | 11 ++++ config.h.in | 6 +++ configure | 151 +++++++++++++++++++++++++++++++++++----------------- configure.ac | 70 +++++++++++++++++------- manual/install.texi | 11 ++++ 6 files changed, 193 insertions(+), 69 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 0dd96f2d0f..f1b95e0e79 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,16 @@ +2016-12-26 Nick Alcock + + [BZ #7065] + * configure.ac (libc_cv_ssp): Move up. + (libc_cv_ssp_strong): Likewise. + (libc_cv_ssp_all): New. + (stack_protector): Augment, adding -fstack-protector-all. + (no_stack_protector): New. + (STACK_PROTECTOR_LEVEL): New. + (AC_ARG_ENABLE(stack-protector)): New configure flag. + * manual/install.texi (--enable-stack-protector): Document it. + * config.h.in (STACK_PROTECTOR_LEVEL): New macro. + 2016-12-24 Carlos O'Donell * README.pretty-printers: Must specify CPPFLAGS-* also. diff --git a/INSTALL b/INSTALL index acb622a102..104f36b0bf 100644 --- a/INSTALL +++ b/INSTALL @@ -135,6 +135,17 @@ will be used, and CFLAGS sets optimization options for the compiler. '--enable-lock-elision=yes' Enable lock elision for pthread mutexes by default. +'--enable-stack-protector' +'--enable-stack-protector=strong' +'--enable-stack-protector=all' + Compile the C library and all other parts of the glibc package + (including the threading and math libraries, NSS modules, and + transliteration modules) using the GCC '-fstack-protector', + '-fstack-protector-strong' or '-fstack-protector-all' options to + detect stack overruns. Only the dynamic linker and a small number + of routines called directly from assembler are excluded from this + protection. + '--enable-pt_chown' The file 'pt_chown' is a helper binary for 'grantpt' (*note Pseudo-Terminals: Allocation.) that is installed setuid root to fix diff --git a/config.h.in b/config.h.in index 33757bd553..d96ce0f8cf 100644 --- a/config.h.in +++ b/config.h.in @@ -48,6 +48,12 @@ /* Define if compiler accepts -ftree-loop-distribute-patterns. */ #undef HAVE_CC_INHIBIT_LOOP_TO_LIBCALL +/* The level of stack protection in use for glibc as a whole. + May be overridden on a file-by-file basis. */ +#ifndef STACK_PROTECTOR_LEVEL +#undef STACK_PROTECTOR_LEVEL +#endif + /* Define if the regparm attribute shall be used for local functions (gcc on ix86 only). */ #undef USE_REGPARMS diff --git a/configure b/configure index 10b0fb7069..8c69f09232 100755 --- a/configure +++ b/configure @@ -620,7 +620,6 @@ libc_cv_cc_loop_to_function libc_cv_cc_submachine libc_cv_cc_nofma libc_cv_mtls_dialect_gnu2 -stack_protector fno_unit_at_a_time libc_cv_output_format libc_cv_has_glob_dat @@ -661,6 +660,9 @@ sysdeps_add_ons sysnames submachine multi_arch +no_stack_protector +stack_protector +libc_cv_ssp base_machine add_on_subdirs add_ons @@ -766,6 +768,7 @@ enable_lock_elision enable_add_ons enable_hidden_plt enable_bind_now +enable_stack_protector enable_static_nss enable_force_install enable_maintainer_mode @@ -1427,6 +1430,9 @@ Optional Features: for add-ons if no parameter given --disable-hidden-plt do not hide internal function calls to avoid PLT --enable-bind-now disable lazy relocations in DSOs + --enable-stack-protector=[yes|no|all|strong] + Use -fstack-protector[-all|-strong] to detect glibc + buffer overflows --enable-static-nss build static NSS modules [default=no] --disable-force-install don't force installation of files from this package, even if they are older than the installed files @@ -3427,6 +3433,18 @@ if test "x$bindnow" = xyes; then fi +# Check whether --enable-stack-protector was given. +if test "${enable_stack_protector+set}" = set; then : + enableval=$enable_stack_protector; enable_stack_protector=$enableval +else + enable_stack_protector=no +fi + +case "$enable_stack_protector" in +all|yes|no|strong) ;; +*) as_fn_error $? "Not a valid argument for --enable-stack-protector: \"$enable_stack_protector\"" "$LINENO" 5;; +esac + # Check whether --enable-static-nss was given. if test "${enable_static_nss+set}" = set; then : enableval=$enable_static_nss; static_nss=$enableval @@ -3912,6 +3930,89 @@ fi test -n "$base_machine" || base_machine=$machine +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for -fstack-protector" >&5 +$as_echo_n "checking for -fstack-protector... " >&6; } +if ${libc_cv_ssp+:} false; then : + $as_echo_n "(cached) " >&6 +else + if { ac_try='${CC-cc} $CFLAGS $CPPFLAGS -Werror -fstack-protector -xc /dev/null -S -o /dev/null' + { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 + (eval $ac_try) 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; }; then : + libc_cv_ssp=yes +else + libc_cv_ssp=no +fi + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_ssp" >&5 +$as_echo "$libc_cv_ssp" >&6; } + +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for -fstack-protector-strong" >&5 +$as_echo_n "checking for -fstack-protector-strong... " >&6; } +if ${libc_cv_ssp_strong+:} false; then : + $as_echo_n "(cached) " >&6 +else + if { ac_try='${CC-cc} $CFLAGS $CPPFLAGS -Werror -fstack-protector-strong -xc /dev/null -S -o /dev/null' + { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 + (eval $ac_try) 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; }; then : + libc_cv_ssp_strong=yes +else + libc_cv_ssp_strong=no +fi + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_ssp_strong" >&5 +$as_echo "$libc_cv_ssp_strong" >&6; } + +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for -fstack-protector-all" >&5 +$as_echo_n "checking for -fstack-protector-all... " >&6; } +if ${libc_cv_ssp_all+:} false; then : + $as_echo_n "(cached) " >&6 +else + if { ac_try='${CC-cc} $CFLAGS $CPPFLAGS -Werror -fstack-protector-all -xc /dev/null -S -o /dev/null' + { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 + (eval $ac_try) 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; }; then : + libc_cv_ssp_all=yes +else + libc_cv_ssp_all=no +fi + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_ssp_all" >&5 +$as_echo "$libc_cv_ssp_all" >&6; } + +stack_protector= +no_stack_protector= +if test "$libc_cv_ssp" = yes; then + no_stack_protector="-fno-stack-protector -DSTACK_PROTECTOR_LEVEL=0" +fi + +if test "$enable_stack_protector" = yes && test "$libc_cv_ssp" = yes; then + stack_protector="-fstack-protector" + $as_echo "#define STACK_PROTECTOR_LEVEL 1" >>confdefs.h + +elif test "$enable_stack_protector" = all && test "$libc_cv_ssp_all" = yes; then + stack_protector="-fstack-protector-all" + $as_echo "#define STACK_PROTECTOR_LEVEL 2" >>confdefs.h + +elif test "$enable_stack_protector" = strong && test "$libc_cv_ssp_strong" = yes; then + stack_protector="-fstack-protector-strong" + $as_echo "#define STACK_PROTECTOR_LEVEL 3" >>confdefs.h + +fi + + + + # For the multi-arch option we need support in the assembler & linker. { $as_echo "$as_me:${as_lineno-$LINENO}: checking for assembler and linker STT_GNU_IFUNC support" >&5 $as_echo_n "checking for assembler and linker STT_GNU_IFUNC support... " >&6; } @@ -5915,54 +6016,6 @@ else fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for -fstack-protector" >&5 -$as_echo_n "checking for -fstack-protector... " >&6; } -if ${libc_cv_ssp+:} false; then : - $as_echo_n "(cached) " >&6 -else - if { ac_try='${CC-cc} $CFLAGS $CPPFLAGS -Werror -fstack-protector -xc /dev/null -S -o /dev/null' - { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 - (eval $ac_try) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; }; then : - libc_cv_ssp=yes -else - libc_cv_ssp=no -fi - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_ssp" >&5 -$as_echo "$libc_cv_ssp" >&6; } - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for -fstack-protector-strong" >&5 -$as_echo_n "checking for -fstack-protector-strong... " >&6; } -if ${libc_cv_ssp_strong+:} false; then : - $as_echo_n "(cached) " >&6 -else - if { ac_try='${CC-cc} $CFLAGS $CPPFLAGS -Werror -fstack-protector-strong -xc /dev/null -S -o /dev/null' - { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 - (eval $ac_try) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; }; then : - libc_cv_ssp_strong=yes -else - libc_cv_ssp_strong=no -fi - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_ssp_strong" >&5 -$as_echo "$libc_cv_ssp_strong" >&6; } - -stack_protector= -if test "$libc_cv_ssp_strong" = "yes"; then - stack_protector="-fstack-protector-strong" -elif test "$libc_cv_ssp" = "yes"; then - stack_protector="-fstack-protector" -fi - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -mtls-dialect=gnu2" >&5 $as_echo_n "checking for -mtls-dialect=gnu2... " >&6; } if ${libc_cv_mtls_dialect_gnu2+:} false; then : diff --git a/configure.ac b/configure.ac index aa6e2d7e56..c159768984 100644 --- a/configure.ac +++ b/configure.ac @@ -235,6 +235,18 @@ if test "x$bindnow" = xyes; then AC_DEFINE(BIND_NOW) fi +dnl Build glibc with -fstack-protector, -fstack-protector-all, or +dnl -fstack-protector-strong. +AC_ARG_ENABLE([stack-protector], + AC_HELP_STRING([--enable-stack-protector=@<:@yes|no|all|strong@:>@], + [Use -fstack-protector[-all|-strong] to detect glibc buffer overflows]), + [enable_stack_protector=$enableval], + [enable_stack_protector=no]) +case "$enable_stack_protector" in +all|yes|no|strong) ;; +*) AC_MSG_ERROR([Not a valid argument for --enable-stack-protector: \"$enable_stack_protector\"]);; +esac + dnl On some platforms we cannot use dynamic loading. We must provide dnl static NSS modules. AC_ARG_ENABLE([static-nss], @@ -619,6 +631,44 @@ fi test -n "$base_machine" || base_machine=$machine AC_SUBST(base_machine) +AC_CACHE_CHECK(for -fstack-protector, libc_cv_ssp, [dnl +LIBC_TRY_CC_OPTION([$CFLAGS $CPPFLAGS -Werror -fstack-protector], + [libc_cv_ssp=yes], + [libc_cv_ssp=no]) +]) + +AC_CACHE_CHECK(for -fstack-protector-strong, libc_cv_ssp_strong, [dnl +LIBC_TRY_CC_OPTION([$CFLAGS $CPPFLAGS -Werror -fstack-protector-strong], + [libc_cv_ssp_strong=yes], + [libc_cv_ssp_strong=no]) +]) + +AC_CACHE_CHECK(for -fstack-protector-all, libc_cv_ssp_all, [dnl +LIBC_TRY_CC_OPTION([$CFLAGS $CPPFLAGS -Werror -fstack-protector-all], + [libc_cv_ssp_all=yes], + [libc_cv_ssp_all=no]) +]) + +stack_protector= +no_stack_protector= +if test "$libc_cv_ssp" = yes; then + no_stack_protector="-fno-stack-protector -DSTACK_PROTECTOR_LEVEL=0" +fi + +if test "$enable_stack_protector" = yes && test "$libc_cv_ssp" = yes; then + stack_protector="-fstack-protector" + AC_DEFINE(STACK_PROTECTOR_LEVEL, 1) +elif test "$enable_stack_protector" = all && test "$libc_cv_ssp_all" = yes; then + stack_protector="-fstack-protector-all" + AC_DEFINE(STACK_PROTECTOR_LEVEL, 2) +elif test "$enable_stack_protector" = strong && test "$libc_cv_ssp_strong" = yes; then + stack_protector="-fstack-protector-strong" + AC_DEFINE(STACK_PROTECTOR_LEVEL, 3) +fi +AC_SUBST(libc_cv_ssp) +AC_SUBST(stack_protector) +AC_SUBST(no_stack_protector) + # For the multi-arch option we need support in the assembler & linker. AC_CACHE_CHECK([for assembler and linker STT_GNU_IFUNC support], libc_cv_ld_gnu_indirect_function, [dnl @@ -1461,26 +1511,6 @@ else fi AC_SUBST(fno_unit_at_a_time) -AC_CACHE_CHECK(for -fstack-protector, libc_cv_ssp, [dnl -LIBC_TRY_CC_OPTION([$CFLAGS $CPPFLAGS -Werror -fstack-protector], - [libc_cv_ssp=yes], - [libc_cv_ssp=no]) -]) - -AC_CACHE_CHECK(for -fstack-protector-strong, libc_cv_ssp_strong, [dnl -LIBC_TRY_CC_OPTION([$CFLAGS $CPPFLAGS -Werror -fstack-protector-strong], - [libc_cv_ssp_strong=yes], - [libc_cv_ssp_strong=no]) -]) - -stack_protector= -if test "$libc_cv_ssp_strong" = "yes"; then - stack_protector="-fstack-protector-strong" -elif test "$libc_cv_ssp" = "yes"; then - stack_protector="-fstack-protector" -fi -AC_SUBST(stack_protector) - AC_CACHE_CHECK([for -mtls-dialect=gnu2], libc_cv_mtls_dialect_gnu2, [dnl cat > conftest.c < Date: Sat, 31 Dec 2016 23:32:17 +0530 Subject: Add framework for tunables The tunables framework allows us to uniformly manage and expose global variables inside glibc as switches to users. tunables/README has instructions for glibc developers to add new tunables. Tunables support can be enabled by passing the --enable-tunables configure flag to the configure script. This patch only adds a framework and does not pose any limitations on how tunable values are read from the user. It also adds environment variables used in malloc behaviour tweaking to the tunables framework as a PoC of the compatibility interface. * manual/install.texi: Add --enable-tunables option. * INSTALL: Regenerate. * README.tunables: New file. * Makeconfig (CPPFLAGS): Define TOP_NAMESPACE. (before-compile): Generate dl-tunable-list.h early. * config.h.in: Add HAVE_TUNABLES. * config.make.in: Add have-tunables. * configure.ac: Add --enable-tunables option. * configure: Regenerate. * csu/init-first.c (__libc_init_first): Move __libc_init_secure earlier... * csu/init-first.c (LIBC_START_MAIN):... to here. Include dl-tunables.h, libc-internal.h. (LIBC_START_MAIN) [!SHARED]: Initialize tunables for static binaries. * elf/Makefile (dl-routines): Add dl-tunables. * elf/Versions (ld): Add __tunable_set_val to GLIBC_PRIVATE namespace. * elf/dl-support (_dl_nondynamic_init): Unset MALLOC_CHECK_ only when !HAVE_TUNABLES. * elf/rtld.c (process_envvars): Likewise. * elf/dl-sysdep.c [HAVE_TUNABLES]: Include dl-tunables.h (_dl_sysdep_start): Call __tunables_init. * elf/dl-tunable-types.h: New file. * elf/dl-tunables.c: New file. * elf/dl-tunables.h: New file. * elf/dl-tunables.list: New file. * malloc/tst-malloc-usable-static.c: New test case. * malloc/Makefile (tests-static): Add it. * malloc/arena.c [HAVE_TUNABLES]: Include dl-tunables.h. Define TUNABLE_NAMESPACE. (DL_TUNABLE_CALLBACK (set_mallopt_check)): New function. (DL_TUNABLE_CALLBACK_FNDECL): New macro. Use it to define callback functions. (ptmalloc_init): Set tunable values. * scripts/gen-tunables.awk: New file. * sysdeps/mach/hurd/dl-sysdep.c: Include dl-tunables.h. (_dl_sysdep_start): Call __tunables_init. --- ChangeLog | 41 +++++ INSTALL | 5 + Makeconfig | 16 ++ README.tunables | 85 ++++++++++ config.h.in | 3 + config.make.in | 1 + configure | 16 ++ configure.ac | 10 ++ csu/init-first.c | 2 - csu/libc-start.c | 8 + elf/Makefile | 5 + elf/Versions | 3 + elf/dl-support.c | 2 + elf/dl-sysdep.c | 4 + elf/dl-tunable-types.h | 46 ++++++ elf/dl-tunables.c | 320 ++++++++++++++++++++++++++++++++++++++ elf/dl-tunables.h | 88 +++++++++++ elf/dl-tunables.list | 69 ++++++++ elf/rtld.c | 2 + malloc/Makefile | 2 + malloc/arena.c | 54 +++++++ malloc/tst-malloc-usable-static.c | 1 + manual/install.texi | 5 + scripts/gen-tunables.awk | 157 +++++++++++++++++++ sysdeps/mach/hurd/dl-sysdep.c | 4 + 25 files changed, 947 insertions(+), 2 deletions(-) create mode 100644 README.tunables create mode 100644 elf/dl-tunable-types.h create mode 100644 elf/dl-tunables.c create mode 100644 elf/dl-tunables.h create mode 100644 elf/dl-tunables.list create mode 100644 malloc/tst-malloc-usable-static.c create mode 100644 scripts/gen-tunables.awk (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index d219e5432b..85a4ac30df 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,44 @@ +2016-12-31 Siddhesh Poyarekar + + * manual/install.texi: Add --enable-tunables option. + * INSTALL: Regenerate. + * README.tunables: New file. + * Makeconfig (CPPFLAGS): Define TOP_NAMESPACE. + (before-compile): Generate dl-tunable-list.h early. + * config.h.in: Add HAVE_TUNABLES. + * config.make.in: Add have-tunables. + * configure.ac: Add --enable-tunables option. + * configure: Regenerate. + * csu/init-first.c (__libc_init_first): Move + __libc_init_secure earlier... + * csu/init-first.c (LIBC_START_MAIN):... to here. + Include dl-tunables.h, libc-internal.h. + (LIBC_START_MAIN) [!SHARED]: Initialize tunables for static + binaries. + * elf/Makefile (dl-routines): Add dl-tunables. + * elf/Versions (ld): Add __tunable_set_val to GLIBC_PRIVATE + namespace. + * elf/dl-support (_dl_nondynamic_init): Unset MALLOC_CHECK_ + only when !HAVE_TUNABLES. + * elf/rtld.c (process_envvars): Likewise. + * elf/dl-sysdep.c [HAVE_TUNABLES]: Include dl-tunables.h + (_dl_sysdep_start): Call __tunables_init. + * elf/dl-tunable-types.h: New file. + * elf/dl-tunables.c: New file. + * elf/dl-tunables.h: New file. + * elf/dl-tunables.list: New file. + * malloc/tst-malloc-usable-static.c: New test case. + * malloc/Makefile (tests-static): Add it. + * malloc/arena.c [HAVE_TUNABLES]: Include dl-tunables.h. + Define TUNABLE_NAMESPACE. + (DL_TUNABLE_CALLBACK (set_mallopt_check)): New function. + (DL_TUNABLE_CALLBACK_FNDECL): New macro. Use it to define + callback functions. + (ptmalloc_init): Set tunable values. + * scripts/gen-tunables.awk: New file. + * sysdeps/mach/hurd/dl-sysdep.c: Include dl-tunables.h. + (_dl_sysdep_start): Call __tunables_init. + 2016-12-31 Florian Weimer * resolv/resolv.h (RES_BLAST): Deprecate. diff --git a/INSTALL b/INSTALL index 104f36b0bf..25619fc520 100644 --- a/INSTALL +++ b/INSTALL @@ -169,6 +169,11 @@ will be used, and CFLAGS sets optimization options for the compiler. By default for x86_64, the GNU C Library is built with the vector math library. Use this option to disable the vector math library. +'--enable-tunables' + Tunables support allows additional library parameters to be + customized at runtime. This is an experimental feature and affects + startup time and is thus disabled by default. + '--build=BUILD-SYSTEM' '--host=HOST-SYSTEM' These options are for cross-compiling. If you specify both options diff --git a/Makeconfig b/Makeconfig index 0158eaa76e..b173e4cc08 100644 --- a/Makeconfig +++ b/Makeconfig @@ -934,6 +934,11 @@ CPPFLAGS = $(config-extra-cppflags) $(CPPUNDEFS) $(CPPFLAGS-config) \ $(foreach lib,$(libof-$(basename $(@F))) \ $(libof-$( $@.tmp + mv $@.tmp $@ +endif + common-generated += libc-modules.h libc-modules.stmp # The name under which the run-time dynamic linker is installed. diff --git a/README.tunables b/README.tunables new file mode 100644 index 0000000000..df74f3b24b --- /dev/null +++ b/README.tunables @@ -0,0 +1,85 @@ + TUNABLE FRAMEWORK + ================= + +Tunables is a feature in the GNU C Library that allows application authors and +distribution maintainers to alter the runtime library behaviour to match their +workload. + +The tunable framework allows modules within glibc to register variables that +may be tweaked through an environment variable. It aims to enforce a strict +namespace rule to bring consistency to naming of these tunable environment +variables across the project. This document is a guide for glibc developers to +add tunables to the framework. + +ADDING A NEW TUNABLE +-------------------- + +The TOP_NAMESPACE macro is defined by default as 'glibc'. If distributions +intend to add their own tunables, they should do so in a different top +namespace by overriding the TOP_NAMESPACE macro for that tunable. Downstream +implementations are discouraged from using the 'glibc' top namespace for +tunables they don't already have consensus to push upstream. + +There are two steps to adding a tunable: + +1. Add a tunable ID: + +Modules that wish to use the tunables interface must define the +TUNABLE_NAMESPACE macro. Following this, for each tunable you want to +add, make an entry in elf/dl-tunables.list. The format of the file is as +follows: + +TOP_NAMESPACE { + NAMESPACE1 { + TUNABLE1 { + # tunable attributes, one per line + } + # A tunable with default attributes, i.e. string variable. + TUNABLE2 + TUNABLE3 { + # its attributes + } + } + NAMESPACE2 { + ... + } +} + +The list of allowed attributes are: + +- type: Data type. Defaults to STRING. Allowed types are: + INT_32, SIZE_T and STRING. + +- minval: Optional minimum acceptable value. For a string type + this is the minimum length of the value. + +- maxval: Optional maximum acceptable value. For a string type + this is the maximum length of the value. + +- env_alias: An alias environment variable + +- is_secure: Specify whether the tunable should be read for setuid + binaries. True allows the tunable to be read for + setuid binaries while false disables it. Note that + even if this is set as true and the value is read, it + may not be used if it does not validate against the + acceptable values or is not considered safe by the + module. + +2. Call either the TUNABLE_SET_VALUE and pass into it the tunable name and a + pointer to the variable that should be set with the tunable value. + If additional work needs to be done after setting the value, use the + TUNABLE_SET_VALUE_WITH_CALLBACK instead and additionally pass a pointer to + the function that should be called if the tunable value has been set. + +FUTURE WORK +----------- + +The framework currently only allows a one-time initialization of variables +through environment variables and in some cases, modification of variables via +an API call. A future goals for this project include: + +- Setting system-wide and user-wide defaults for tunables through some + mechanism like a configuration file. + +- Allow tweaking of some tunables at runtime diff --git a/config.h.in b/config.h.in index 82f95a6dbd..7bfe923c06 100644 --- a/config.h.in +++ b/config.h.in @@ -256,4 +256,7 @@ /* PowerPC32 uses fctidz for floating point to long long conversions. */ #define HAVE_PPC_FCTIDZ 0 +/* Build glibc with tunables support. */ +#define HAVE_TUNABLES 0 + #endif diff --git a/config.make.in b/config.make.in index 4422025e59..2f8dae213d 100644 --- a/config.make.in +++ b/config.make.in @@ -96,6 +96,7 @@ use-nscd = @use_nscd@ build-hardcoded-path-in-tests= @hardcoded_path_in_tests@ build-pt-chown = @build_pt_chown@ enable-lock-elision = @enable_lock_elision@ +have-tunables = @have_tunables@ # Build tools. CC = @CC@ diff --git a/configure b/configure index c88f6fe88c..d80d738fd1 100755 --- a/configure +++ b/configure @@ -666,6 +666,7 @@ libc_cv_ssp base_machine add_on_subdirs add_ons +have_tunables build_pt_chown build_nscd link_obsolete_rpc @@ -782,6 +783,7 @@ enable_systemtap enable_build_nscd enable_nscd enable_pt_chown +enable_tunables enable_mathvec with_cpu ' @@ -1452,6 +1454,7 @@ Optional Features: --disable-build-nscd disable building and installing the nscd daemon --disable-nscd library functions will not contact the nscd daemon --enable-pt_chown Enable building and installing pt_chown + --enable-tunables Enable tunables support --enable-mathvec Enable building and installing mathvec [default depends on architecture] @@ -3698,6 +3701,19 @@ if test "$build_pt_chown" = yes; then fi +# Check whether --enable-tunables was given. +if test "${enable_tunables+set}" = set; then : + enableval=$enable_tunables; have_tunables=$enableval +else + have_tunables=no +fi + + +if test "$have_tunables" = yes; then + $as_echo "#define HAVE_TUNABLES 1" >>confdefs.h + +fi + # The abi-tags file uses a fairly simplistic model for name recognition that # can't distinguish i486-pc-linux-gnu fully from i486-pc-gnu. So we mutate a # $host_os of `gnu*' here to be `gnu-gnu*' just so that it can tell. diff --git a/configure.ac b/configure.ac index 2782bfaf08..22f5cab200 100644 --- a/configure.ac +++ b/configure.ac @@ -421,6 +421,16 @@ if test "$build_pt_chown" = yes; then AC_DEFINE(HAVE_PT_CHOWN) fi +AC_ARG_ENABLE([tunables], + [AS_HELP_STRING([--enable-tunables], + [Enable tunables support])], + [have_tunables=$enableval], + [have_tunables=no]) +AC_SUBST(have_tunables) +if test "$have_tunables" = yes; then + AC_DEFINE(HAVE_TUNABLES) +fi + # The abi-tags file uses a fairly simplistic model for name recognition that # can't distinguish i486-pc-linux-gnu fully from i486-pc-gnu. So we mutate a # $host_os of `gnu*' here to be `gnu-gnu*' just so that it can tell. diff --git a/csu/init-first.c b/csu/init-first.c index 77c6e1cb9e..465f25b722 100644 --- a/csu/init-first.c +++ b/csu/init-first.c @@ -72,8 +72,6 @@ _init (int argc, char **argv, char **envp) __environ = envp; #ifndef SHARED - __libc_init_secure (); - /* First the initialization which normally would be done by the dynamic linker. */ _dl_non_dynamic_init (); diff --git a/csu/libc-start.c b/csu/libc-start.c index cc59073abe..15db9b4684 100644 --- a/csu/libc-start.c +++ b/csu/libc-start.c @@ -21,6 +21,9 @@ #include #include #include +#include + +#include extern void __libc_init_first (int argc, char **argv, char **envp); @@ -174,6 +177,11 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), } } + /* Initialize very early so that tunables can use it. */ + __libc_init_secure (); + + __tunables_init (__environ); + /* Perform IREL{,A} relocations. */ apply_irel (); diff --git a/elf/Makefile b/elf/Makefile index 8a2ce02cd5..de28d99224 100644 --- a/elf/Makefile +++ b/elf/Makefile @@ -35,6 +35,11 @@ dl-routines = $(addprefix dl-,load lookup object reloc deps hwcaps \ ifeq (yes,$(use-ldconfig)) dl-routines += dl-cache endif + +ifeq (yes,$(have-tunables)) +dl-routines += dl-tunables +endif + all-dl-routines = $(dl-routines) $(sysdep-dl-routines) # But they are absent from the shared libc, because that code is in ld.so. elide-routines.os = $(all-dl-routines) dl-support enbl-secure dl-origin \ diff --git a/elf/Versions b/elf/Versions index 3d57e36fd2..6abe9db8bd 100644 --- a/elf/Versions +++ b/elf/Versions @@ -70,5 +70,8 @@ ld { # Internal error handling support. Interposed by libc.so. _dl_signal_error; _dl_catch_error; + + # Set value of a tunable. + __tunable_set_val; } } diff --git a/elf/dl-support.c b/elf/dl-support.c index c30194c7b7..d350d6d0d0 100644 --- a/elf/dl-support.c +++ b/elf/dl-support.c @@ -354,8 +354,10 @@ _dl_non_dynamic_init (void) cp = (const char *) __rawmemchr (cp, '\0') + 1; } +#if !HAVE_TUNABLES if (__access ("/etc/suid-debug", F_OK) != 0) __unsetenv ("MALLOC_CHECK_"); +#endif } #ifdef DL_PLATFORM_INIT diff --git a/elf/dl-sysdep.c b/elf/dl-sysdep.c index eaa71556d2..4283767fe0 100644 --- a/elf/dl-sysdep.c +++ b/elf/dl-sysdep.c @@ -44,6 +44,8 @@ #include #include +#include + extern char **_environ attribute_hidden; extern char _end[] attribute_hidden; @@ -219,6 +221,8 @@ _dl_sysdep_start (void **start_argptr, } #endif + __tunables_init (_environ); + #ifdef DL_SYSDEP_INIT DL_SYSDEP_INIT; #endif diff --git a/elf/dl-tunable-types.h b/elf/dl-tunable-types.h new file mode 100644 index 0000000000..d1591b6efb --- /dev/null +++ b/elf/dl-tunable-types.h @@ -0,0 +1,46 @@ +/* Tunable type information. + + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _TUNABLE_TYPES_H_ +# define _TUNABLE_TYPES_H_ +#include + +typedef void (*tunable_callback_t) (void *); + +typedef enum +{ + TUNABLE_TYPE_INT_32, + TUNABLE_TYPE_SIZE_T, + TUNABLE_TYPE_STRING +} tunable_type_code_t; + +typedef struct +{ + tunable_type_code_t type_code; + int64_t min; + int64_t max; +} tunable_type_t; + +typedef union +{ + int64_t numval; + const char *strval; +} tunable_val_t; + +#endif diff --git a/elf/dl-tunables.c b/elf/dl-tunables.c new file mode 100644 index 0000000000..472747b3da --- /dev/null +++ b/elf/dl-tunables.c @@ -0,0 +1,320 @@ +/* The tunable framework. See the README.tunables to know how to use the + tunable in a glibc module. + + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#define TUNABLES_INTERNAL 1 +#include "dl-tunables.h" + +/* Compare environment names, bounded by the name hardcoded in glibc. */ +static bool +is_name (const char *orig, const char *envname) +{ + for (;*orig != '\0' && *envname != '\0'; envname++, orig++) + if (*orig != *envname) + break; + + /* The ENVNAME is immediately followed by a value. */ + if (*orig == '\0' && *envname == '=') + return true; + else + return false; +} + +static char ** +get_next_env (char **envp, char **name, size_t *namelen, char **val) +{ + while (envp != NULL && *envp != NULL) + { + char *envline = *envp; + int len = 0; + + while (envline[len] != '\0' && envline[len] != '=') + len++; + + /* Just the name and no value, go to the next one. */ + if (envline[len] == '\0') + continue; + + *name = envline; + *namelen = len; + *val = &envline[len + 1]; + + return ++envp; + } + + return NULL; +} + +static int +tunables_unsetenv (char **ep, const char *name) +{ + while (*ep != NULL) + { + size_t cnt = 0; + + while ((*ep)[cnt] == name[cnt] && name[cnt] != '\0') + ++cnt; + + if (name[cnt] == '\0' && (*ep)[cnt] == '=') + { + /* Found it. Remove this pointer by moving later ones to + the front. */ + char **dp = ep; + + do + dp[0] = dp[1]; + while (*dp++); + /* Continue the loop in case NAME appears again. */ + } + else + ++ep; + } + + return 0; +} + +/* A stripped down strtoul-like implementation for very early use. It does not + set errno if the result is outside bounds because it gets called before + errno may have been set up. */ +static unsigned long int +tunables_strtoul (const char *nptr) +{ + unsigned long int result = 0; + long int sign = 1; + unsigned max_digit; + + while (*nptr == ' ' || *nptr == '\t') + ++nptr; + + if (*nptr == '-') + { + sign = -1; + ++nptr; + } + else if (*nptr == '+') + ++nptr; + + if (*nptr < '0' || *nptr > '9') + return 0UL; + + int base = 10; + max_digit = 9; + if (*nptr == '0') + { + if (nptr[1] == 'x' || nptr[1] == 'X') + { + base = 16; + nptr += 2; + } + else + { + base = 8; + max_digit = 7; + } + } + + while (1) + { + unsigned long int digval; + if (*nptr >= '0' && *nptr <= '0' + max_digit) + digval = *nptr - '0'; + else if (base == 16) + { + if (*nptr >= 'a' && *nptr <= 'f') + digval = *nptr - 'a' + 10; + else if (*nptr >= 'A' && *nptr <= 'F') + digval = *nptr - 'A' + 10; + else + break; + } + else + break; + + if (result > ULONG_MAX / base + || (result == ULONG_MAX / base && digval > ULONG_MAX % base)) + return ULONG_MAX; + result *= base; + result += digval; + ++nptr; + } + + return result * sign; +} + +/* Initialize the internal type if the value validates either using the + explicit constraints of the tunable or with the implicit constraints of its + type. */ +static void +tunable_set_val_if_valid_range (tunable_t *cur, const char *strval, + int64_t default_min, int64_t default_max) +{ + int64_t val = tunables_strtoul (strval); + + int64_t min = cur->type.min; + int64_t max = cur->type.max; + + if (min == max) + { + min = default_min; + max = default_max; + } + + if (val >= min && val <= max) + { + cur->val.numval = val; + cur->strval = strval; + } +} + +/* Validate range of the input value and initialize the tunable CUR if it looks + good. */ +static void +tunable_initialize (tunable_t *cur, const char *strval) +{ + switch (cur->type.type_code) + { + case TUNABLE_TYPE_INT_32: + { + tunable_set_val_if_valid_range (cur, strval, INT32_MIN, INT32_MAX); + break; + } + case TUNABLE_TYPE_SIZE_T: + { + tunable_set_val_if_valid_range (cur, strval, 0, SIZE_MAX); + break; + } + case TUNABLE_TYPE_STRING: + { + cur->val.strval = cur->strval = strval; + break; + } + default: + __builtin_unreachable (); + } +} + +/* Disable a tunable if it is set. */ +static void +disable_tunable (tunable_id_t id, char **envp) +{ + const char *env_alias = tunable_list[id].env_alias; + + if (env_alias != NULL) + tunables_unsetenv (envp, tunable_list[id].env_alias); +} + +/* Disable the glibc.malloc.check tunable in SETUID/SETGID programs unless + the system administrator overrides it by creating the /etc/suid-debug + file. This is a special case where we want to conditionally enable/disable + a tunable even for setuid binaries. We use the special version of access() + to avoid setting ERRNO, which is a TLS variable since TLS has not yet been + set up. */ +static inline void +__always_inline +maybe_disable_malloc_check (void) +{ + if (__libc_enable_secure && __access_noerrno ("/etc/suid-debug", F_OK) != 0) + disable_tunable (TUNABLE_ENUM_NAME(glibc, malloc, check), __environ); +} + +/* Initialize the tunables list from the environment. For now we only use the + ENV_ALIAS to find values. Later we will also use the tunable names to find + values. */ +void +__tunables_init (char **envp) +{ + char *envname = NULL; + char *envval = NULL; + size_t len = 0; + + maybe_disable_malloc_check (); + + while ((envp = get_next_env (envp, &envname, &len, &envval)) != NULL) + { + for (int i = 0; i < sizeof (tunable_list) / sizeof (tunable_t); i++) + { + tunable_t *cur = &tunable_list[i]; + + /* Skip over tunables that have either been set already or should be + skipped. */ + if (cur->strval != NULL || cur->env_alias == NULL + || (__libc_enable_secure && !cur->is_secure)) + continue; + + const char *name = cur->env_alias; + + /* We have a match. Initialize and move on to the next line. */ + if (is_name (name, envname)) + { + tunable_initialize (cur, envval); + break; + } + } + } +} + +/* Set the tunable value. This is called by the module that the tunable exists + in. */ +void +__tunable_set_val (tunable_id_t id, void *valp, tunable_callback_t callback) +{ + tunable_t *cur = &tunable_list[id]; + + /* Don't do anything if our tunable was not set during initialization or if + it failed validation. */ + if (cur->strval == NULL) + return; + + if (valp == NULL) + goto cb; + + switch (cur->type.type_code) + { + case TUNABLE_TYPE_INT_32: + { + *((int32_t *) valp) = (int32_t) cur->val.numval; + break; + } + case TUNABLE_TYPE_SIZE_T: + { + *((size_t *) valp) = (size_t) cur->val.numval; + break; + } + case TUNABLE_TYPE_STRING: + { + *((const char **)valp) = cur->val.strval; + break; + } + default: + __builtin_unreachable (); + } + +cb: + if (callback) + callback (&cur->val); +} diff --git a/elf/dl-tunables.h b/elf/dl-tunables.h new file mode 100644 index 0000000000..a3f5472b45 --- /dev/null +++ b/elf/dl-tunables.h @@ -0,0 +1,88 @@ +/* The tunable framework. See the README to know how to use the tunable in + a glibc module. + + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _TUNABLES_H_ +#define _TUNABLES_H_ + +#if !HAVE_TUNABLES +static inline void +__always_inline +__tunables_init (char **unused __attribute_unused) +{ + /* This is optimized out if tunables are not enabled. */ +} +#else + +# include +# include "dl-tunable-types.h" + +/* A tunable. */ +struct _tunable +{ + const char *name; /* Internal name of the tunable. */ + tunable_type_t type; /* Data type of the tunable. */ + tunable_val_t val; /* The value. */ + const char *strval; /* The string containing the value, + points into envp. */ + bool is_secure; /* Whether the tunable must be read + even for setuid binaries. Note that + even if the tunable is read, it may + not get used by the target module if + the value is considered unsafe. */ + /* Compatibility elements. */ + const char *env_alias; /* The compatibility environment + variable name. */ +}; + +typedef struct _tunable tunable_t; + +/* Full name for a tunable is top_ns.tunable_ns.id. */ +# define TUNABLE_NAME_S(top,ns,id) #top "." #ns "." #id + +# define TUNABLE_ENUM_NAME(top,ns,id) TUNABLE_ENUM_NAME1 (top,ns,id) +# define TUNABLE_ENUM_NAME1(top,ns,id) top ## _ ## ns ## _ ## id + +# include "dl-tunable-list.h" + +extern void __tunables_init (char **); +extern void __tunable_set_val (tunable_id_t, void *, tunable_callback_t); + +/* Check if the tunable has been set to a non-default value and if it is, copy + it over into __VAL. */ +# define TUNABLE_SET_VAL(__id,__val) \ +({ \ + __tunable_set_val \ + (TUNABLE_ENUM_NAME (TOP_NAMESPACE, TUNABLE_NAMESPACE, __id), (__val), \ + NULL); \ +}) + +/* Same as TUNABLE_SET_VAL, but also call the callback function __CB. */ +# define TUNABLE_SET_VAL_WITH_CALLBACK(__id,__val,__cb) \ +({ \ + __tunable_set_val \ + (TUNABLE_ENUM_NAME (TOP_NAMESPACE, TUNABLE_NAMESPACE, __id), (__val), \ + DL_TUNABLE_CALLBACK (__cb)); \ +}) + +/* Namespace sanity for callback functions. Use this macro to keep the + namespace of the modules clean. */ +# define DL_TUNABLE_CALLBACK(__name) _dl_tunable_ ## __name +#endif +#endif diff --git a/elf/dl-tunables.list b/elf/dl-tunables.list new file mode 100644 index 0000000000..11504c49e3 --- /dev/null +++ b/elf/dl-tunables.list @@ -0,0 +1,69 @@ +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. + +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. + +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. + +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +# Allowed attributes for tunables: +# +# type: Defaults to STRING +# minval: Optional minimum acceptable value +# maxval: Optional maximum acceptable value +# env_alias: An alias environment variable +# is_secure: Specify whether the environment variable should be read for +# setuid binaries. + +glibc { + malloc { + check { + type: INT_32 + minval: 0 + maxval: 3 + env_alias: MALLOC_CHECK_ + is_secure: true + } + top_pad { + type: SIZE_T + env_alias: MALLOC_TOP_PAD_ + } + perturb { + type: INT_32 + minval: 0 + maxval: 0xff + env_alias: MALLOC_PERTURB_ + } + mmap_threshold { + type: SIZE_T + env_alias: MALLOC_MMAP_THRESHOLD_ + } + trim_threshold { + type: SIZE_T + env_alias: MALLOC_TRIM_THRESHOLD_ + } + mmap_max { + type: INT_32 + env_alias: MALLOC_MMAP_MAX_ + } + arena_max { + type: SIZE_T + env_alias: MALLOC_ARENA_MAX + minval: 1 + } + arena_test { + type: SIZE_T + env_alias: MALLOC_ARENA_TEST + minval: 1 + } + } +} diff --git a/elf/rtld.c b/elf/rtld.c index 4ec25d7c30..a60ead693b 100644 --- a/elf/rtld.c +++ b/elf/rtld.c @@ -2510,7 +2510,9 @@ process_envvars (enum mode *modep) if (__access ("/etc/suid-debug", F_OK) != 0) { +#if !HAVE_TUNABLES unsetenv ("MALLOC_CHECK_"); +#endif GLRO(dl_debug_mask) = 0; } diff --git a/malloc/Makefile b/malloc/Makefile index b8efcd68bc..4e4104ec8b 100644 --- a/malloc/Makefile +++ b/malloc/Makefile @@ -37,6 +37,7 @@ tests := mallocbug tst-malloc tst-valloc tst-calloc tst-obstack \ tests-static := \ tst-interpose-static-nothread \ tst-interpose-static-thread \ + tst-malloc-usable-static tests += $(tests-static) test-srcs = tst-mtrace @@ -158,6 +159,7 @@ endif tst-mcheck-ENV = MALLOC_CHECK_=3 tst-malloc-usable-ENV = MALLOC_CHECK_=3 +tst-malloc-usable-static-ENV = $(tst-malloc-usable-ENV) # Uncomment this for test releases. For public releases it is too expensive. #CPPFLAGS-malloc.o += -DMALLOC_DEBUG=1 diff --git a/malloc/arena.c b/malloc/arena.c index eed42471a7..234035f14f 100644 --- a/malloc/arena.c +++ b/malloc/arena.c @@ -19,6 +19,11 @@ #include +#if HAVE_TUNABLES +# define TUNABLE_NAMESPACE malloc +#endif +#include + /* Compile-time constants. */ #define HEAP_MIN_SIZE (32 * 1024) @@ -204,6 +209,34 @@ __malloc_fork_unlock_child (void) __libc_lock_init (list_lock); } +#if HAVE_TUNABLES +static inline int do_set_mallopt_check (int32_t value); +void +DL_TUNABLE_CALLBACK (set_mallopt_check) (void *valp) +{ + int32_t value = *(int32_t *) valp; + do_set_mallopt_check (value); + if (check_action != 0) + __malloc_check_init (); +} + +# define DL_TUNABLE_CALLBACK_FNDECL(__name, __type) \ +static inline int do_ ## __name (__type value); \ +void \ +DL_TUNABLE_CALLBACK (__name) (void *valp) \ +{ \ + __type value = *(__type *) valp; \ + do_ ## __name (value); \ +} + +DL_TUNABLE_CALLBACK_FNDECL (set_mmap_threshold, size_t) +DL_TUNABLE_CALLBACK_FNDECL (set_mmaps_max, int32_t) +DL_TUNABLE_CALLBACK_FNDECL (set_top_pad, size_t) +DL_TUNABLE_CALLBACK_FNDECL (set_perturb_byte, int32_t) +DL_TUNABLE_CALLBACK_FNDECL (set_trim_threshold, size_t) +DL_TUNABLE_CALLBACK_FNDECL (set_arena_max, size_t) +DL_TUNABLE_CALLBACK_FNDECL (set_arena_test, size_t) +#else /* Initialization routine. */ #include extern char **_environ; @@ -238,6 +271,7 @@ next_env_entry (char ***position) return result; } +#endif #ifdef SHARED @@ -272,6 +306,24 @@ ptmalloc_init (void) #endif thread_arena = &main_arena; + +#if HAVE_TUNABLES + /* Ensure initialization/consolidation and do it under a lock so that a + thread attempting to use the arena in parallel waits on us till we + finish. */ + __libc_lock_lock (main_arena.mutex); + malloc_consolidate (&main_arena); + + TUNABLE_SET_VAL_WITH_CALLBACK (check, NULL, set_mallopt_check); + TUNABLE_SET_VAL_WITH_CALLBACK (top_pad, NULL, set_top_pad); + TUNABLE_SET_VAL_WITH_CALLBACK (perturb, NULL, set_perturb_byte); + TUNABLE_SET_VAL_WITH_CALLBACK (mmap_threshold, NULL, set_mmap_threshold); + TUNABLE_SET_VAL_WITH_CALLBACK (trim_threshold, NULL, set_trim_threshold); + TUNABLE_SET_VAL_WITH_CALLBACK (mmap_max, NULL, set_mmaps_max); + TUNABLE_SET_VAL_WITH_CALLBACK (arena_max, NULL, set_arena_max); + TUNABLE_SET_VAL_WITH_CALLBACK (arena_test, NULL, set_arena_test); + __libc_lock_unlock (main_arena.mutex); +#else const char *s = NULL; if (__glibc_likely (_environ != NULL)) { @@ -340,6 +392,8 @@ ptmalloc_init (void) if (check_action != 0) __malloc_check_init (); } +#endif + #if HAVE_MALLOC_INIT_HOOK void (*hook) (void) = atomic_forced_read (__malloc_initialize_hook); if (hook != NULL) diff --git a/malloc/tst-malloc-usable-static.c b/malloc/tst-malloc-usable-static.c new file mode 100644 index 0000000000..8907db01a5 --- /dev/null +++ b/malloc/tst-malloc-usable-static.c @@ -0,0 +1 @@ +#include diff --git a/manual/install.texi b/manual/install.texi index d02e87091f..d41296294e 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -200,6 +200,11 @@ configure with @option{--disable-werror}. By default for x86_64, @theglibc{} is built with the vector math library. Use this option to disable the vector math library. +@item --enable-tunables +Tunables support allows additional library parameters to be customized at +runtime. This is an experimental feature and affects startup time and is thus +disabled by default. + @item --build=@var{build-system} @itemx --host=@var{host-system} These options are for cross-compiling. If you specify both options and diff --git a/scripts/gen-tunables.awk b/scripts/gen-tunables.awk new file mode 100644 index 0000000000..b65b5a4a33 --- /dev/null +++ b/scripts/gen-tunables.awk @@ -0,0 +1,157 @@ +# Generate dl-tunable-list.h from dl-tunables.list + +BEGIN { + tunable="" + ns="" + top_ns="" +} + +# Skip over blank lines and comments. +/^#/ { + next +} + +/^[ \t]*$/ { + next +} + +# Beginning of either a top namespace, tunable namespace or a tunable, decided +# on the current value of TUNABLE, NS or TOP_NS. +$2 == "{" { + if (top_ns == "") { + top_ns = $1 + } + else if (ns == "") { + ns = $1 + } + else if (tunable == "") { + tunable = $1 + } + else { + printf ("Unexpected occurrence of '{': %s:%d\n", FILENAME, FNR) + exit 1 + } + + next +} + +# End of either a top namespace, tunable namespace or a tunable. +$1 == "}" { + if (tunable != "") { + # Tunables definition ended, now fill in default attributes. + if (!types[top_ns][ns][tunable]) { + types[top_ns][ns][tunable] = "STRING" + } + if (!minvals[top_ns][ns][tunable]) { + minvals[top_ns][ns][tunable] = "0" + } + if (!maxvals[top_ns][ns][tunable]) { + maxvals[top_ns][ns][tunable] = "0" + } + if (!env_alias[top_ns][ns][tunable]) { + env_alias[top_ns][ns][tunable] = "NULL" + } + if (!is_secure[top_ns][ns][tunable]) { + is_secure[top_ns][ns][tunable] = "false" + } + + tunable = "" + } + else if (ns != "") { + ns = "" + } + else if (top_ns != "") { + top_ns = "" + } + else { + printf ("syntax error: extra }: %s:%d\n", FILENAME, FNR) + exit 1 + } + next +} + +# Everything else, which could either be a tunable without any attributes or a +# tunable attribute. +{ + if (ns == "") { + printf("Line %d: Invalid tunable outside a namespace: %s\n", NR, $0) + exit 1 + } + + if (tunable == "") { + # We encountered a tunable without any attributes, so note it with a + # default. + types[top_ns][ns][$1] = "STRING" + next + } + + # Otherwise, we have encountered a tunable attribute. + split($0, arr, ":") + attr = gensub(/^[ \t]+|[ \t]+$/, "", "g", arr[1]) + val = gensub(/^[ \t]+|[ \t]+$/, "", "g", arr[2]) + + if (attr == "type") { + types[top_ns][ns][tunable] = val + } + else if (attr == "minval") { + minvals[top_ns][ns][tunable] = val + } + else if (attr == "maxval") { + maxvals[top_ns][ns][tunable] = val + } + else if (attr == "env_alias") { + env_alias[top_ns][ns][tunable] = sprintf("\"%s\"", val) + } + else if (attr == "is_secure") { + if (val == "true" || val == "false") { + is_secure[top_ns][ns][tunable] = val + } + else { + printf("Line %d: Invalid value (%s) for is_secure: %s, ", NR, val, + $0) + print("Allowed values are 'true' or 'false'") + exit 1 + } + } +} + +END { + if (ns != "") { + print "Unterminated namespace. Is a closing brace missing?" + exit 1 + } + + print "/* AUTOGENERATED by gen-tunables.awk. */" + print "#ifndef _TUNABLES_H_" + print "# error \"Do not include this file directly.\"" + print "# error \"Include tunables.h instead.\"" + print "#endif" + + # Now, the enum names + print "\ntypedef enum" + print "{" + for (t in types) { + for (n in types[t]) { + for (m in types[t][n]) { + printf (" TUNABLE_ENUM_NAME(%s, %s, %s),\n", t, n, m); + } + } + } + print "} tunable_id_t;\n" + + # Finally, the tunable list. + print "\n#ifdef TUNABLES_INTERNAL" + print "static tunable_t tunable_list[] = {" + for (t in types) { + for (n in types[t]) { + for (m in types[t][n]) { + printf (" {TUNABLE_NAME_S(%s, %s, %s)", t, n, m) + printf (", {TUNABLE_TYPE_%s, %s, %s}, {.numval = 0}, NULL, %s, %s},\n", + types[t][n][m], minvals[t][n][m], maxvals[t][n][m], + is_secure[t][n][m], env_alias[t][n][m]); + } + } + } + print "};" + print "#endif" +} diff --git a/sysdeps/mach/hurd/dl-sysdep.c b/sysdeps/mach/hurd/dl-sysdep.c index d730f82280..42bccdab79 100644 --- a/sysdeps/mach/hurd/dl-sysdep.c +++ b/sysdeps/mach/hurd/dl-sysdep.c @@ -44,6 +44,8 @@ #include #include +#include + extern void __mach_init (void); extern int _dl_argc; @@ -143,6 +145,8 @@ _dl_sysdep_start (void **start_argptr, __libc_enable_secure = _dl_hurd_data->flags & EXEC_SECURE; + __tunables_init (_environ); + if (_dl_hurd_data->flags & EXEC_STACK_ARGS && _dl_hurd_data->user_entry == 0) _dl_hurd_data->user_entry = (vm_address_t) ENTRY_POINT; -- cgit v1.2.3 From 6765d5d34d126b26d55e2d73dac4dfec5e6d6241 Mon Sep 17 00:00:00 2001 From: Siddhesh Poyarekar Date: Sat, 31 Dec 2016 23:34:04 +0530 Subject: Enhance --enable-tunables to select tunables frontend at build time At the GNU Tools Cauldron 2016, the state of the current tunables patchset was considered OK with the addition of a way to select the frontend to be used for the tunables. That is, to avoid being locked in to one type of frontend initially, it should be possible to build tunables with a different frontend with something as simple as a configure switch. To that effect, this patch enhances the --enable-tunables option to accept more values than just 'yes' or 'no'. The current frontend (and default when enable-tunables is 'yes') is called 'valstring', to select the frontend where a single environment variable is set to a colon-separated value string. More such frontends can be added in future. * Makeconfig (have-tunables): Check for non-negative instead of positive. * configure.ac: Add 'valstring' as a valid value for --enable-tunables. * configure: Regenerate. * elf/Makefile (have-tunables): Check for non-negative instead of positive. (CPPFLAGS-dl-tunables.c): Define TUNABLES_FRONTEND for dl-tunables.c. * elf/dl-tunables.c (GLIBC_TUNABLES): Define only when TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring. (tunables_strdup): Likewise. (disable_tunables): Likewise. (parse_tunables): Likewise. (__tunables_init): Process GLIBC_TUNABLES envvar only when. TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring. * elf/dl-tunables.h (TUNABLES_FRONTEND_valstring): New macro. (TUNABLES_FRONTEND_yes): New macro, define as TUNABLES_FRONTEND_valstring by default. * manual/install.texi: Document new acceptable values for --enable-tunables. * INSTALL: Regenerate. --- ChangeLog | 23 +++++++++++++++++++++++ INSTALL | 18 +++++++++++++++++- Makeconfig | 4 ++-- configure | 3 ++- configure.ac | 2 +- elf/Makefile | 4 +++- elf/dl-tunables.c | 12 +++++++++++- elf/dl-tunables.h | 4 ++++ manual/install.texi | 17 ++++++++++++++++- 9 files changed, 79 insertions(+), 8 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 69e527c3ea..eded27fbe3 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,28 @@ 2016-12-31 Siddhesh Poyarekar + * Makeconfig (have-tunables): Check for non-negative instead + of positive. + * configure.ac: Add 'valstring' as a valid value for + --enable-tunables. + * configure: Regenerate. + * elf/Makefile (have-tunables): Check for non-negative instead + of positive. + (CPPFLAGS-dl-tunables.c): Define TUNABLES_FRONTEND for + dl-tunables.c. + * elf/dl-tunables.c (GLIBC_TUNABLES): Define only when + TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring. + (tunables_strdup): Likewise. + (disable_tunables): Likewise. + (parse_tunables): Likewise. + (__tunables_init): Process GLIBC_TUNABLES envvar only when. + TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring. + * elf/dl-tunables.h (TUNABLES_FRONTEND_valstring): New macro. + (TUNABLES_FRONTEND_yes): New macro, define as + TUNABLES_FRONTEND_valstring by default. + * manual/install.texi: Document new acceptable values for + --enable-tunables. + * INSTALL: Regenerate. + * config.make.in (have-loop-to-function): Define. * elf/Makefile (CFLAGS-dl-tunables.c): Add -fno-tree-loop-distribute-patterns. diff --git a/INSTALL b/INSTALL index 25619fc520..55d52c5f15 100644 --- a/INSTALL +++ b/INSTALL @@ -172,7 +172,23 @@ will be used, and CFLAGS sets optimization options for the compiler. '--enable-tunables' Tunables support allows additional library parameters to be customized at runtime. This is an experimental feature and affects - startup time and is thus disabled by default. + startup time and is thus disabled by default. This option can take + the following values: + + 'no' + This is the default if the option is not passed to configure. + This disables tunables. + + 'yes' + This is the default if the option is passed to configure. + This enables tunables and selects the default frontend + (currently 'valstring'). + + 'valstring' + This enables tunables and selects the 'valstring' frontend for + tunables. This frontend allows users to specify tunables as a + colon-separated list in a single environment variable + 'GLIBC_TUNABLES'. '--build=BUILD-SYSTEM' '--host=HOST-SYSTEM' diff --git a/Makeconfig b/Makeconfig index b173e4cc08..1a2db6da1d 100644 --- a/Makeconfig +++ b/Makeconfig @@ -935,7 +935,7 @@ CPPFLAGS = $(config-extra-cppflags) $(CPPUNDEFS) $(CPPFLAGS-config) \ $(libof-$( Date: Wed, 18 Jan 2017 17:27:47 +0000 Subject: Update install.texi latest GCC version known to work. * manual/install.texi (Tools for Compilation): Update GCC version known to work to build glibc. * INSTALL: Regenerated. --- ChangeLog | 6 ++++++ INSTALL | 2 +- manual/install.texi | 2 +- 3 files changed, 8 insertions(+), 2 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 424f8b9108..6c78761a5f 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2017-01-18 Joseph Myers + + * manual/install.texi (Tools for Compilation): Update GCC version + known to work to build glibc. + * INSTALL: Regenerated. + 2017-01-17 Stefan Liebler [BZ #21006] diff --git a/INSTALL b/INSTALL index 55d52c5f15..3b3fd121b2 100644 --- a/INSTALL +++ b/INSTALL @@ -415,7 +415,7 @@ build the GNU C Library: GCC 4.7 or higher is required. In general it is recommended to use the newest version of the compiler that is known to work for building the GNU C Library, as newer compilers usually produce - better code. As of release time, GCC 5.3 is the newest compiler + better code. As of release time, GCC 6.3 is the newest compiler verified to work to build the GNU C Library. For multi-arch support it is recommended to use a GCC which has diff --git a/manual/install.texi b/manual/install.texi index 2657da108b..266add8ba9 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -460,7 +460,7 @@ GCC 4.7 or newer GCC 4.7 or higher is required. In general it is recommended to use the newest version of the compiler that is known to work for building @theglibc{}, as newer compilers usually produce better code. As of -release time, GCC 5.3 is the newest compiler verified to work to build +release time, GCC 6.3 is the newest compiler verified to work to build @theglibc{}. For multi-arch support it is recommended to use a GCC which has been built with -- cgit v1.2.3 From 2d6ab5df3b675e96ee587ae6a8c2ce004c6b1ba9 Mon Sep 17 00:00:00 2001 From: Florian Weimer Date: Thu, 2 Mar 2017 14:44:28 +0100 Subject: Document and fix --enable-bind-now [BZ #21015] --- ChangeLog | 17 +++++++++++++++++ INSTALL | 6 ++++++ Makeconfig | 7 +++++++ Makerules | 9 +++------ iconvdata/Makefile | 5 +++++ manual/install.texi | 6 ++++++ sysdeps/unix/sysv/linux/alpha/localplt.data | 2 +- sysdeps/unix/sysv/linux/i386/localplt.data | 2 +- sysdeps/x86_64/localplt.data | 2 +- 9 files changed, 47 insertions(+), 9 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index a587b9a2a5..bed3bd1e5c 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,20 @@ +2017-03-02 Florian Weimer + + [BZ #21015] + * manual/install.texi (Configuring and compiling): Document + --enable-bind-now. + * Makeconfig [bind-now] (LDFLAGS-lib.so): Set. + (build-shlib-helper): Use $(LDFLAGS-lib.so). + (format.lds): Likewise. + [bind-now] (LDFLAGS-c.so): Remove. + * sysdeps/x86_64/localplt.data (libm.so): matherr relocation can + be R_X86_64_GLOB_DAT. + * sysdeps/unix/sysv/linux/i386/localplt.data (libm.so): matherr + relocation can be R_386_GLOB_DAT. + * sysdeps/unix/sysv/linux/alpha/localplt.data (libm.so): matherr + relocaiton can be R_ALPHA_GLOB_DAT. + * iconvdata/Makefile [bind-now] (LDFLAGS.so): Add -Wl,-z,now. + 2017-03-01 Zack Weinberg * include/libc-pointer-arith.h: New file. Define diff --git a/INSTALL b/INSTALL index 3b3fd121b2..e77cb2d4e2 100644 --- a/INSTALL +++ b/INSTALL @@ -146,6 +146,12 @@ will be used, and CFLAGS sets optimization options for the compiler. of routines called directly from assembler are excluded from this protection. +'--enable-bind-now' + Disable lazy binding for installed shared objects. This provides + additional security hardening because it enables full RELRO and a + read-only global offset table (GOT), at the cost of slightly + increased program load times. + '--enable-pt_chown' The file 'pt_chown' is a helper binary for 'grantpt' (*note Pseudo-Terminals: Allocation.) that is installed setuid root to fix diff --git a/Makeconfig b/Makeconfig index 97a15b569e..1c815113b9 100644 --- a/Makeconfig +++ b/Makeconfig @@ -386,6 +386,13 @@ LDFLAGS.so += $(hashstyle-LDFLAGS) LDFLAGS-rtld += $(hashstyle-LDFLAGS) endif +# If lazy relocations are disabled, add the -z now flag. Use +# LDFLAGS-lib.so instead of LDFLAGS.so, to avoid adding the flag to +# test modules. +ifeq ($(bind-now),yes) +LDFLAGS-lib.so += -Wl,-z,now +endif + # Command to run after every final link (executable or shared object). # This is invoked with $(call after-link,...), so it should operate on # the file $1. This can be set to do some sort of post-processing on diff --git a/Makerules b/Makerules index e9194e54cf..7f0eef8096 100644 --- a/Makerules +++ b/Makerules @@ -588,7 +588,7 @@ $(LINK.o) -shared -static-libgcc -Wl,-O1 $(sysdep-LDFLAGS) \ $(extra-B-$(@F:lib%.so=%).so) -B$(csu-objpfx) \ $(extra-B-$(@F:lib%.so=%).so) $(load-map-file) \ -Wl,-soname=lib$(libprefix)$(@F:lib%.so=%).so$($(@F)-version) \ - $(LDFLAGS.so) $(LDFLAGS-$(@F:lib%.so=%).so) \ + $(LDFLAGS.so) $(LDFLAGS-lib.so) $(LDFLAGS-$(@F:lib%.so=%).so) \ -L$(subst :, -L,$(rpath-link)) -Wl,-rpath-link=$(rpath-link) endef @@ -686,10 +686,6 @@ LDFLAGS-c.so = -nostdlib -nostartfiles LDLIBS-c.so += $(libc.so-gnulib) # Give libc.so an entry point and make it directly runnable itself. LDFLAGS-c.so += -e __libc_main -# If lazy relocation is disabled add the -z now flag. -ifeq ($(bind-now),yes) -LDFLAGS-c.so += -Wl,-z,now -endif # Pre-link the objects of libc_pic.a so that we can locally resolve # COMMON symbols before we link against ld.so. This is because ld.so # contains some of libc_pic.a already, which will prevent the COMMONs @@ -1104,7 +1100,8 @@ $(common-objpfx)format.lds: $(..)scripts/output-format.sed \ ifneq (unknown,$(output-format)) echo > $@.new 'OUTPUT_FORMAT($(output-format))' else - $(LINK.o) -shared $(sysdep-LDFLAGS) $(rtld-LDFLAGS) $(LDFLAGS.so) \ + $(LINK.o) -shared $(sysdep-LDFLAGS) $(rtld-LDFLAGS) \ + $(LDFLAGS.so) $(LDFLAGS-lib.so) \ -x c /dev/null -o $@.so -Wl,--verbose -v 2>&1 \ | sed -n -f $< > $@.new test -s $@.new diff --git a/iconvdata/Makefile b/iconvdata/Makefile index 04157b25c5..e4845871f5 100644 --- a/iconvdata/Makefile +++ b/iconvdata/Makefile @@ -63,6 +63,11 @@ modules := ISO8859-1 ISO8859-2 ISO8859-3 ISO8859-4 ISO8859-5 \ MAC-CENTRALEUROPE KOI8-RU ISO8859-9E \ CP770 CP771 CP772 CP773 CP774 +# If lazy binding is disabled, use BIND_NOW for the gconv modules. +ifeq ($(bind-now),yes) +LDFLAGS.so += -Wl,-z,now +endif + modules.so := $(addsuffix .so, $(modules)) ifeq (yes,$(build-shared)) diff --git a/manual/install.texi b/manual/install.texi index 266add8ba9..3398cfab02 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -175,6 +175,12 @@ options to detect stack overruns. Only the dynamic linker and a small number of routines called directly from assembler are excluded from this protection. +@item --enable-bind-now +Disable lazy binding for installed shared objects. This provides +additional security hardening because it enables full RELRO and a +read-only global offset table (GOT), at the cost of slightly increased +program load times. + @pindex pt_chown @findex grantpt @item --enable-pt_chown diff --git a/sysdeps/unix/sysv/linux/alpha/localplt.data b/sysdeps/unix/sysv/linux/alpha/localplt.data index cca17f1e34..1f0e3b494e 100644 --- a/sysdeps/unix/sysv/linux/alpha/localplt.data +++ b/sysdeps/unix/sysv/linux/alpha/localplt.data @@ -20,7 +20,7 @@ libc.so: free + RELA R_ALPHA_GLOB_DAT libc.so: malloc + RELA R_ALPHA_GLOB_DAT libc.so: memalign + RELA R_ALPHA_GLOB_DAT libc.so: realloc + RELA R_ALPHA_GLOB_DAT -libm.so: matherr +libm.so: matherr + RELA R_ALPHA_GLOB_DAT # We used to offer inline functions that used this, so it must be exported. # Ought to reorg things such that carg isn't thus forced to use a plt. libm.so: __atan2 diff --git a/sysdeps/unix/sysv/linux/i386/localplt.data b/sysdeps/unix/sysv/linux/i386/localplt.data index 2c2584956d..8ea4333846 100644 --- a/sysdeps/unix/sysv/linux/i386/localplt.data +++ b/sysdeps/unix/sysv/linux/i386/localplt.data @@ -6,7 +6,7 @@ libc.so: free + REL R_386_GLOB_DAT libc.so: malloc + REL R_386_GLOB_DAT libc.so: memalign + REL R_386_GLOB_DAT libc.so: realloc + REL R_386_GLOB_DAT -libm.so: matherr +libm.so: matherr + REL R_386_GLOB_DAT # The main malloc is interposed into the dynamic linker, for # allocations after the initial link (when dlopen is used). ld.so: malloc + REL R_386_GLOB_DAT diff --git a/sysdeps/x86_64/localplt.data b/sysdeps/x86_64/localplt.data index 014a9f4554..a1840cff31 100644 --- a/sysdeps/x86_64/localplt.data +++ b/sysdeps/x86_64/localplt.data @@ -8,7 +8,7 @@ libc.so: free + RELA R_X86_64_GLOB_DAT libc.so: malloc + RELA R_X86_64_GLOB_DAT libc.so: memalign + RELA R_X86_64_GLOB_DAT libc.so: realloc + RELA R_X86_64_GLOB_DAT -libm.so: matherr +libm.so: matherr + RELA R_X86_64_GLOB_DAT # The main malloc is interposed into the dynamic linker, for # allocations after the initial link (when dlopen is used). ld.so: malloc + RELA R_X86_64_GLOB_DAT -- cgit v1.2.3 From a640393a18329ef4044bf9213f6466cd2d1e69f3 Mon Sep 17 00:00:00 2001 From: Joseph Myers Date: Tue, 21 Mar 2017 16:40:16 +0000 Subject: Regenerate INSTALL. --- ChangeLog | 4 ++++ INSTALL | 6 ++++++ 2 files changed, 10 insertions(+) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index c292a5bf29..3ed2df8719 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,7 @@ +2017-03-21 Joseph Myers + + * INSTALL: Regenerated. + 2017-03-21 Thorsten Kukuk * config.h.in: Add LINK_OBSOLETE_NSL. diff --git a/INSTALL b/INSTALL index e77cb2d4e2..60f714e0df 100644 --- a/INSTALL +++ b/INSTALL @@ -196,6 +196,12 @@ will be used, and CFLAGS sets optimization options for the compiler. colon-separated list in a single environment variable 'GLIBC_TUNABLES'. +'--enable-obsolete-nsl' + By default, libnsl is only built as shared library for backward + compatibility and the NSS modules libnss_compat, libnss_nis and + libnss_nisplus are not built at all. Use this option to enable + libnsl with all depending NSS modules and header files. + '--build=BUILD-SYSTEM' '--host=HOST-SYSTEM' These options are for cross-compiling. If you specify both options -- cgit v1.2.3 From e92030239abb4038d4f915d47021d6c037239309 Mon Sep 17 00:00:00 2001 From: Florian Weimer Date: Wed, 19 Apr 2017 07:44:48 +0200 Subject: Assume that accept4 is always available and works Simplify the Linux accept4 implementation based on the assumption that it is available in some way. __ASSUME_ACCEPT4_SOCKETCALL was previously unused, so remove it. For ia64, the accept4 system call (and socket call) were backported in kernel version 3.2.18. Reflect this in the installation instructions. --- ChangeLog | 32 ++++++ INSTALL | 16 +-- manual/install.texi | 2 + nscd/connections.c | 50 +--------- socket/Makefile | 2 + socket/tst-accept4.c | 131 +++++++++++++++++++++++++ support/Makefile | 1 + support/xaccept4.c | 32 ++++++ support/xsocket.h | 1 + sysdeps/mach/hurd/kernel-features.h | 2 - sysdeps/nacl/kernel-features.h | 4 - sysdeps/unix/sysv/linux/accept4.c | 69 ++----------- sysdeps/unix/sysv/linux/ia64/configure | 4 + sysdeps/unix/sysv/linux/ia64/configure.ac | 4 + sysdeps/unix/sysv/linux/ia64/kernel-features.h | 7 +- sysdeps/unix/sysv/linux/kernel-features.h | 11 +-- 16 files changed, 230 insertions(+), 138 deletions(-) create mode 100644 socket/tst-accept4.c create mode 100644 support/xaccept4.c (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index fc8c2e4f7b..28fcdb24c7 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,35 @@ +2017-04-19 Florian Weimer + + * nscd/connections.c (have_accept4): Removed definition. + (nscd_run_worker, main_loop_poll, main_loop_epolll): Assume that + accept4 works. + * manual/install.texi (Linux): Require at least kernel 3.2.18 for + ia64 because that was the first version with accept4 support. + * support/Makefile (libsupport-routines): Add xaccept4. + * support/xsocket.h (xaccept4): Declare. + * support/xaccept4.c: New file. + * socket/tst-accept4.c: New file. + * socket/Makefile (tests): Add tst-accept4. + * sysdeps/mach/hurd/kernel-features.h (__ASSUME_ACCEPT4): Remove + definition. + * sysdeps/nacl/kernel-features.h (__ASSUME_ACCEPT4): Remove + comment. + * sysdeps/unix/sysv/linux/accept4.c: Assume that an accept4 + implementation is available. + (accept4): Use the system call if available, otherwise use the + socket call. + * sysdeps/unix/sysv/linux/ia64/configure.ac (arch_minimum_kernel): + Set to 3.2.18. + * sysdeps/unix/sysv/linux/ia64/kernel-features.h + (__ASSUME_ACCEPT4_SYSCALL, __ASSUME_ACCEPT4): Do not undefine. + accept4 is unconditionally available in later 3.2 stable kernels. + (__ASSUME_ACCEPT4_SYSCALL): Define. + * sysdeps/unix/sysv/linux/kernel-features.h + (__ASSUME_ACCEPT4_SOCKETCALL, __ASSUME_ACCEPT4): Remove + definitions. + * sysdeps/unix/sysv/linux/i386/kernel-features.h + (__ASSUME_ACCEPT4_SYSCALL): Define for Linux 4.3 or later. + 2017-04-18 Joseph Myers * conform/Makefile (tests-special): Do not make addition of diff --git a/INSTALL b/INSTALL index 60f714e0df..920c4df0ef 100644 --- a/INSTALL +++ b/INSTALL @@ -501,13 +501,15 @@ Specific advice for GNU/Linux systems If you are installing the GNU C Library on GNU/Linux systems, you need to have the header files from a 3.2 or newer kernel around for -reference. These headers must be installed using 'make -headers_install'; the headers present in the kernel source directory are -not suitable for direct use by the GNU C Library. You do not need to -use that kernel, just have its headers installed where the GNU C Library -can access them, referred to here as INSTALL-DIRECTORY. The easiest way -to do this is to unpack it in a directory such as -'/usr/src/linux-VERSION'. In that directory, run 'make headers_install +reference. (For the ia64 architecture, you need version 3.2.18 or newer +because this is the first version with support for the 'accept4' system +call.) These headers must be installed using 'make headers_install'; +the headers present in the kernel source directory are not suitable for +direct use by the GNU C Library. You do not need to use that kernel, +just have its headers installed where the GNU C Library can access them, +referred to here as INSTALL-DIRECTORY. The easiest way to do this is to +unpack it in a directory such as '/usr/src/linux-VERSION'. In that +directory, run 'make headers_install INSTALL_HDR_PATH=INSTALL-DIRECTORY'. Finally, configure the GNU C Library with the option '--with-headers=INSTALL-DIRECTORY/include'. Use the most recent kernel you can get your hands on. (If you are diff --git a/manual/install.texi b/manual/install.texi index 99397c2ee0..d39d2daacd 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -566,6 +566,8 @@ patches, although we try to avoid this. If you are installing @theglibc{} on @gnulinuxsystems{}, you need to have the header files from a 3.2 or newer kernel around for reference. +(For the ia64 architecture, you need version 3.2.18 or newer because this +is the first version with support for the @code{accept4} system call.) These headers must be installed using @samp{make headers_install}; the headers present in the kernel source directory are not suitable for direct use by @theglibc{}. You do not need to use that kernel, just have diff --git a/nscd/connections.c b/nscd/connections.c index a5ca57aea5..cc1ed72077 100644 --- a/nscd/connections.c +++ b/nscd/connections.c @@ -257,10 +257,6 @@ int inotify_fd = -1; static int nl_status_fd = -1; #endif -#ifndef __ASSUME_ACCEPT4 -static int have_accept4; -#endif - /* Number of times clients had to wait. */ unsigned long int client_queued; @@ -1650,16 +1646,6 @@ nscd_run_worker (void *p) /* We are done with the list. */ pthread_mutex_unlock (&readylist_lock); -#ifndef __ASSUME_ACCEPT4 - if (have_accept4 < 0) - { - /* We do not want to block on a short read or so. */ - int fl = fcntl (fd, F_GETFL); - if (fl == -1 || fcntl (fd, F_SETFL, fl | O_NONBLOCK) == -1) - goto close_and_out; - } -#endif - /* Now read the request. */ request_header req; if (__builtin_expect (TEMP_FAILURE_RETRY (read (fd, &req, sizeof (req))) @@ -2099,24 +2085,8 @@ main_loop_poll (void) if (conns[0].revents != 0) { /* We have a new incoming connection. Accept the connection. */ - int fd; - -#ifndef __ASSUME_ACCEPT4 - fd = -1; - if (have_accept4 >= 0) -#endif - { - fd = TEMP_FAILURE_RETRY (accept4 (sock, NULL, NULL, + int fd = TEMP_FAILURE_RETRY (accept4 (sock, NULL, NULL, SOCK_NONBLOCK)); -#ifndef __ASSUME_ACCEPT4 - if (have_accept4 == 0) - have_accept4 = fd != -1 || errno != ENOSYS ? 1 : -1; -#endif - } -#ifndef __ASSUME_ACCEPT4 - if (have_accept4 < 0) - fd = TEMP_FAILURE_RETRY (accept (sock, NULL, NULL)); -#endif /* Use the descriptor if we have not reached the limit. */ if (fd >= 0) @@ -2284,24 +2254,8 @@ main_loop_epoll (int efd) if (revs[cnt].data.fd == sock) { /* A new connection. */ - int fd; - -# ifndef __ASSUME_ACCEPT4 - fd = -1; - if (have_accept4 >= 0) -# endif - { - fd = TEMP_FAILURE_RETRY (accept4 (sock, NULL, NULL, + int fd = TEMP_FAILURE_RETRY (accept4 (sock, NULL, NULL, SOCK_NONBLOCK)); -# ifndef __ASSUME_ACCEPT4 - if (have_accept4 == 0) - have_accept4 = fd != -1 || errno != ENOSYS ? 1 : -1; -# endif - } -# ifndef __ASSUME_ACCEPT4 - if (have_accept4 < 0) - fd = TEMP_FAILURE_RETRY (accept (sock, NULL, NULL)); -# endif /* Use the descriptor if we have not reached the limit. */ if (fd >= 0) diff --git a/socket/Makefile b/socket/Makefile index 25d4f68578..1e2555d66d 100644 --- a/socket/Makefile +++ b/socket/Makefile @@ -31,6 +31,8 @@ routines := accept bind connect getpeername getsockname getsockopt \ setsockopt shutdown socket socketpair isfdtype opensock \ sockatmark accept4 recvmmsg sendmmsg +tests := tst-accept4 + aux := sa_len include ../Rules diff --git a/socket/tst-accept4.c b/socket/tst-accept4.c new file mode 100644 index 0000000000..a5cf3a237b --- /dev/null +++ b/socket/tst-accept4.c @@ -0,0 +1,131 @@ +/* Test the accept4 function with differing flags arguments. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include +#include +#include +#include +#include + +static bool +is_nonblocking (int fd) +{ + int status = fcntl (fd, F_GETFL); + if (status < 0) + FAIL_EXIT1 ("fcntl (F_GETFL): %m"); + return status & O_NONBLOCK; +} + +static bool +is_cloexec (int fd) +{ + int status = fcntl (fd, F_GETFD); + if (status < 0) + FAIL_EXIT1 ("fcntl (F_GETFD): %m"); + return status & FD_CLOEXEC; +} + +struct client +{ + int socket; + struct sockaddr_in address; +}; + +/* Perform a non-blocking connect to *SERVER_ADDRESS. */ +static struct client +client_connect (const struct sockaddr_in *server_address) +{ + struct client result; + result.socket = xsocket (AF_INET, + SOCK_STREAM | SOCK_NONBLOCK | SOCK_CLOEXEC, 0); + TEST_VERIFY (is_nonblocking (result.socket)); + TEST_VERIFY (is_cloexec (result.socket)); + int ret = connect (result.socket, (const struct sockaddr *) server_address, + sizeof (*server_address)); + if (ret < 0 && errno != EINPROGRESS) + FAIL_EXIT1 ("client connect: %m"); + socklen_t sa_len = sizeof (result.address); + xgetsockname (result.socket, (struct sockaddr *) &result.address, + &sa_len); + TEST_VERIFY (sa_len == sizeof (result.address)); + return result; +} + +static void +check_same_address (const struct sockaddr_in *left, + const struct sockaddr_in *right) +{ + TEST_VERIFY (left->sin_family == AF_INET); + TEST_VERIFY (right->sin_family == AF_INET); + TEST_VERIFY (left->sin_addr.s_addr == right->sin_addr.s_addr); + TEST_VERIFY (left->sin_port == right->sin_port); +} + +static int +do_test (void) +{ + /* Create server socket. */ + int server_socket = xsocket (AF_INET, SOCK_STREAM, 0); + TEST_VERIFY (!is_nonblocking (server_socket)); + TEST_VERIFY (!is_cloexec (server_socket)); + struct sockaddr_in server_address = + { + .sin_family = AF_INET, + .sin_addr = {.s_addr = htonl (INADDR_LOOPBACK) }, + }; + xbind (server_socket, + (struct sockaddr *) &server_address, sizeof (server_address)); + { + socklen_t sa_len = sizeof (server_address); + xgetsockname (server_socket, (struct sockaddr *) &server_address, + &sa_len); + TEST_VERIFY (sa_len == sizeof (server_address)); + } + xlisten (server_socket, 5); + + for (int do_nonblock = 0; do_nonblock < 2; ++do_nonblock) + for (int do_cloexec = 0; do_cloexec < 2; ++do_cloexec) + { + int sockflags = 0; + if (do_nonblock) + sockflags |= SOCK_NONBLOCK; + if (do_cloexec) + sockflags |= SOCK_CLOEXEC; + + struct client client = client_connect (&server_address); + struct sockaddr_in client_address; + socklen_t sa_len = sizeof (client_address); + int client_socket = xaccept4 (server_socket, + (struct sockaddr *) &client_address, + &sa_len, sockflags); + TEST_VERIFY (sa_len == sizeof (client_address)); + TEST_VERIFY (is_nonblocking (client_socket) == do_nonblock); + TEST_VERIFY (is_cloexec (client_socket) == do_cloexec); + check_same_address (&client.address, &client_address); + xclose (client_socket); + xclose (client.socket); + } + + xclose (server_socket); + return 0; +} + +#include diff --git a/support/Makefile b/support/Makefile index 1f33fa0c74..38dbd832e9 100644 --- a/support/Makefile +++ b/support/Makefile @@ -50,6 +50,7 @@ libsupport-routines = \ temp_file \ write_message \ xaccept \ + xaccept4 \ xasprintf \ xbind \ xcalloc \ diff --git a/support/xaccept4.c b/support/xaccept4.c new file mode 100644 index 0000000000..67dd95e9fb --- /dev/null +++ b/support/xaccept4.c @@ -0,0 +1,32 @@ +/* accept4 with error checking. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#include +#include +#include + +int +xaccept4 (int fd, struct sockaddr *sa, socklen_t *salen, int flags) +{ + int clientfd = accept4 (fd, sa, salen, flags); + if (clientfd < 0) + FAIL_EXIT1 ("accept4 (%d, 0x%x): %m", fd, flags); + return clientfd; +} diff --git a/support/xsocket.h b/support/xsocket.h index 0dbf13ace9..d6724948d8 100644 --- a/support/xsocket.h +++ b/support/xsocket.h @@ -30,6 +30,7 @@ void xconnect (int, const struct sockaddr *, socklen_t); void xbind (int, const struct sockaddr *, socklen_t); void xlisten (int, int); int xaccept (int, struct sockaddr *, socklen_t *); +int xaccept4 (int, struct sockaddr *, socklen_t *, int); void xsendto (int, const void *, size_t, int, const struct sockaddr *, socklen_t); size_t xrecvfrom (int, void *, size_t, int, struct sockaddr *, socklen_t *); diff --git a/sysdeps/mach/hurd/kernel-features.h b/sysdeps/mach/hurd/kernel-features.h index 20cb3ba7bf..196638829f 100644 --- a/sysdeps/mach/hurd/kernel-features.h +++ b/sysdeps/mach/hurd/kernel-features.h @@ -19,5 +19,3 @@ /* This file can define __ASSUME_* macros checked by certain source files. Almost none of these are used outside of sysdeps/unix/sysv/linux code. But those referring to POSIX-level features like O_* flags can be. */ - -#define __ASSUME_ACCEPT4 1 diff --git a/sysdeps/nacl/kernel-features.h b/sysdeps/nacl/kernel-features.h index 32834fd792..ba315b6638 100644 --- a/sysdeps/nacl/kernel-features.h +++ b/sysdeps/nacl/kernel-features.h @@ -19,7 +19,3 @@ /* This file can define __ASSUME_* macros checked by certain source files. Almost none of these are used outside of sysdeps/unix/sysv/linux code. But those referring to POSIX-level features like O_* flags can be. */ - -/* -#define __ASSUME_ACCEPT4 1 -*/ diff --git a/sysdeps/unix/sysv/linux/accept4.c b/sysdeps/unix/sysv/linux/accept4.c index 53afd7a510..0592f43dd6 100644 --- a/sysdeps/unix/sysv/linux/accept4.c +++ b/sysdeps/unix/sysv/linux/accept4.c @@ -22,75 +22,20 @@ #include #include +#include #include +int +accept4 (int fd, __SOCKADDR_ARG addr, socklen_t *addr_len, int flags) +{ /* Do not use the accept4 syscall on socketcall architectures unless it was added at the same time as the socketcall support or can be assumed to be present. */ #if defined __ASSUME_SOCKETCALL \ && !defined __ASSUME_ACCEPT4_SYSCALL_WITH_SOCKETCALL \ && !defined __ASSUME_ACCEPT4_SYSCALL -# undef __NR_accept4 -#endif - -#ifdef __NR_accept4 -int -accept4 (int fd, __SOCKADDR_ARG addr, socklen_t *addr_len, int flags) -{ - return SYSCALL_CANCEL (accept4, fd, addr.__sockaddr__, addr_len, flags); -} -#elif defined __NR_socketcall -# include -# ifdef __ASSUME_ACCEPT4_SOCKETCALL -int -accept4 (int fd, __SOCKADDR_ARG addr, socklen_t *addr_len, int flags) -{ return SOCKETCALL_CANCEL (accept4, fd, addr.__sockaddr__, addr_len, flags); -} -# else -static int have_accept4; - -int -accept4 (int fd, __SOCKADDR_ARG addr, socklen_t *addr_len, int flags) -{ - if (__glibc_likely (have_accept4 >= 0)) - { - int ret = SOCKETCALL_CANCEL (accept4, fd, addr.__sockaddr__, addr_len, - flags); - /* The kernel returns -EINVAL for unknown socket operations. - We need to convert that error to an ENOSYS error. */ - if (__builtin_expect (ret < 0, 0) - && have_accept4 == 0 - && errno == EINVAL) - { - /* Try another call, this time with the FLAGS parameter - cleared and an invalid file descriptor. This call will not - cause any harm and it will return immediately. */ - ret = SOCKETCALL_CANCEL (invalid, -1); - if (errno == EINVAL) - { - have_accept4 = -1; - __set_errno (ENOSYS); - } - else - { - have_accept4 = 1; - __set_errno (EINVAL); - } - return -1; - } - return ret; - } - __set_errno (ENOSYS); - return -1; -} -# endif /* __ASSUME_ACCEPT4_SOCKETCALL */ -#else /* __NR_socketcall */ -int -accept4 (int fd, __SOCKADDR_ARG addr, socklen_t *addr_len, int flags) -{ - __set_errno (ENOSYS); - return -1; -} -stub_warning (accept4) +#else + return SYSCALL_CANCEL (accept4, fd, addr.__sockaddr__, addr_len, flags); #endif +} diff --git a/sysdeps/unix/sysv/linux/ia64/configure b/sysdeps/unix/sysv/linux/ia64/configure index 1d4e5d18d6..3cf72371ef 100644 --- a/sysdeps/unix/sysv/linux/ia64/configure +++ b/sysdeps/unix/sysv/linux/ia64/configure @@ -2,3 +2,7 @@ # Local configure fragment for sysdeps/unix/sysv/linux/ia64 ldd_rewrite_script=$dir/ldd-rewrite.sed + +# First version with support for the accept4 system call. +# Linux 3.3 includes it as well. +arch_minimum_kernel=3.2.18 diff --git a/sysdeps/unix/sysv/linux/ia64/configure.ac b/sysdeps/unix/sysv/linux/ia64/configure.ac index 4fb564721b..94a578c12d 100644 --- a/sysdeps/unix/sysv/linux/ia64/configure.ac +++ b/sysdeps/unix/sysv/linux/ia64/configure.ac @@ -2,3 +2,7 @@ GLIBC_PROVIDES dnl See aclocal.m4 in the top level source directory. # Local configure fragment for sysdeps/unix/sysv/linux/ia64 ldd_rewrite_script=$dir/ldd-rewrite.sed + +# First version with support for the accept4 system call. +# Linux 3.3 includes it as well. +arch_minimum_kernel=3.2.18 diff --git a/sysdeps/unix/sysv/linux/ia64/kernel-features.h b/sysdeps/unix/sysv/linux/ia64/kernel-features.h index ac9403e869..cda0ad6150 100644 --- a/sysdeps/unix/sysv/linux/ia64/kernel-features.h +++ b/sysdeps/unix/sysv/linux/ia64/kernel-features.h @@ -22,13 +22,8 @@ #include_next -/* Support for the accept4 syscall was added in 3.3. */ -#if __LINUX_KERNEL_VERSION < 0x030300 -# undef __ASSUME_ACCEPT4_SYSCALL -# undef __ASSUME_ACCEPT4 -#endif - #define __ASSUME_RECV_SYSCALL 1 #define __ASSUME_SEND_SYSCALL 1 +#define __ASSUME_ACCEPT4_SYSCALL 1 #endif /* _KERNEL_FEATURES_H */ diff --git a/sysdeps/unix/sysv/linux/kernel-features.h b/sysdeps/unix/sysv/linux/kernel-features.h index fd936c5366..b981466f34 100644 --- a/sysdeps/unix/sysv/linux/kernel-features.h +++ b/sysdeps/unix/sysv/linux/kernel-features.h @@ -75,18 +75,11 @@ architectures using a separate syscall rather than socketcall that syscall was only added later, and some architectures first had socketcall support then a separate syscall. Define - __ASSUME_ACCEPT4_SOCKETCALL if glibc uses socketcall on this - architecture and accept4 is available through socketcall, __ASSUME_ACCEPT4_SYSCALL if it is available through a separate - syscall, __ASSUME_ACCEPT4_SYSCALL_WITH_SOCKETCALL if it became + syscall, and __ASSUME_ACCEPT4_SYSCALL_WITH_SOCKETCALL if it became available through a separate syscall at the same time as through - socketcall, and __ASSUME_ACCEPT4 if the accept4 function is known - to work. */ -#ifdef __ASSUME_SOCKETCALL -# define __ASSUME_ACCEPT4_SOCKETCALL 1 -#endif + socketcall. */ #define __ASSUME_ACCEPT4_SYSCALL 1 -#define __ASSUME_ACCEPT4 1 /* Support for the FUTEX_CLOCK_REALTIME flag was added in 2.6.29. */ #define __ASSUME_FUTEX_CLOCK_REALTIME 1 -- cgit v1.2.3 From 2c0b90ab443abc967cbf75add4f7fde84978cb95 Mon Sep 17 00:00:00 2001 From: Siddhesh Poyarekar Date: Thu, 15 Jun 2017 15:12:54 +0530 Subject: Enable tunables by default All of the major architectures are adopting tunables as a way to add tuning to the library, from hwcap_mask for aarch64 to HLE for s390 and ifunc and cache geometry for x86. Given this adoption and the fact that we don't want additional tuning knobs to be added outside of tunables, it makes sense to enable tunables by default using this trivial patch. Smoke tested on x86 to ensure that tunables code was built without specifying it as a configure flag. I have kept it as --enabled and not changed it to --disable since we want to still keep the option of different kinds of front-ends for tunables. * configure.ac(--enable-tunables): Enable by default. * configure: Regenerate. * NEWS: Mention change. * manual/install.texi (enable-tunables): Adjust documentation. * INSTALL: Regenerate. --- ChangeLog | 8 ++++++++ INSTALL | 18 ++++++++---------- NEWS | 3 +++ configure | 2 +- configure.ac | 2 +- manual/install.texi | 13 ++++++------- 6 files changed, 27 insertions(+), 19 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index a2d1f5ade1..833f03ea6f 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,11 @@ +2017-06-15 Siddhesh Poyarekar + + * configure.ac(--enable-tunables): Enable by default. + * configure: Regenerate. + * NEWS: Mention change. + * manual/install.texi (enable-tunables): Adjust documentation. + * INSTALL: Regenerate. + 2017-06-14 Adhemerval Zanella [BZ #18988] diff --git a/INSTALL b/INSTALL index 920c4df0ef..a2f5a40b17 100644 --- a/INSTALL +++ b/INSTALL @@ -177,18 +177,16 @@ will be used, and CFLAGS sets optimization options for the compiler. '--enable-tunables' Tunables support allows additional library parameters to be - customized at runtime. This is an experimental feature and affects - startup time and is thus disabled by default. This option can take - the following values: - - 'no' - This is the default if the option is not passed to configure. - This disables tunables. + customized at runtime. This feature is enabled by default. This + option can take the following values: 'yes' - This is the default if the option is passed to configure. - This enables tunables and selects the default frontend - (currently 'valstring'). + This is the default if no option is passed to configure. This + enables tunables and selects the default frontend (currently + 'valstring'). + + 'no' + This option disables tunables. 'valstring' This enables tunables and selects the 'valstring' frontend for diff --git a/NEWS b/NEWS index 8d35bd6c95..804c1b92e6 100644 --- a/NEWS +++ b/NEWS @@ -95,6 +95,9 @@ Version 2.26 as atomic variables to try to implement Dekker's mutual exclusion algorithm). +* The tunables feature is now enabled by default. This allows users to tweak + behavior of the GNU C Library using the GLIBC_TUNABLES environment variable. + Security related changes: * The DNS stub resolver limits the advertised UDP buffer size to 1200 bytes, diff --git a/configure b/configure index 4c2922d882..8390f2b86e 100755 --- a/configure +++ b/configure @@ -3725,7 +3725,7 @@ fi if test "${enable_tunables+set}" = set; then : enableval=$enable_tunables; have_tunables=$enableval else - have_tunables=no + have_tunables=yes fi diff --git a/configure.ac b/configure.ac index 3f486d6df4..4c0a18edd9 100644 --- a/configure.ac +++ b/configure.ac @@ -436,7 +436,7 @@ AC_ARG_ENABLE([tunables], [AS_HELP_STRING([--enable-tunables], [Enable tunables support. Known values are 'yes', 'no' and 'valstring'])], [have_tunables=$enableval], - [have_tunables=no]) + [have_tunables=yes]) AC_SUBST(have_tunables) if test "$have_tunables" = yes; then AC_DEFINE(HAVE_TUNABLES) diff --git a/manual/install.texi b/manual/install.texi index d39d2daacd..cbc912817a 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -208,18 +208,17 @@ Use this option to disable the vector math library. @item --enable-tunables Tunables support allows additional library parameters to be customized at -runtime. This is an experimental feature and affects startup time and is thus -disabled by default. This option can take the following values: +runtime. This feature is enabled by default. This option can take the +following values: @table @code -@item no -This is the default if the option is not passed to configure. This disables -tunables. - @item yes -This is the default if the option is passed to configure. This enables tunables +This is the default if no option is passed to configure. This enables tunables and selects the default frontend (currently @samp{valstring}). +@item no +This option disables tunables. + @item valstring This enables tunables and selects the @samp{valstring} frontend for tunables. This frontend allows users to specify tunables as a colon-separated list in a -- cgit v1.2.3 From 4add86749a31f302674599b69d2eea691d69341a Mon Sep 17 00:00:00 2001 From: Joseph Myers Date: Mon, 19 Jun 2017 16:33:00 +0000 Subject: Require GCC 4.9 or later for building glibc. This patch makes configure require GCC 4.9 or later for building glibc, and documents that requirement. Requiring GCC 4.9 or later allows use of _Generic (as in tzcode). It would allow and _Atomic to be used as well if desired, although we need to avoid any libatomic dependencies on any platforms. This patch is explicitly the minimum to implement a new version requirement, with any consequent cleanups of conditional code (not in installed headers or files shared with gnulib etc.) to be done separately. Tested for x86_64. * configure.ac (libc_cv_compiler_ok): Require GCC 4.9 or later. * configure: Regenerated. * manual/install.texi (Tools for Compilation): Document requirement for GCC 4.9 or later. * INSTALL: Regenerated. --- ChangeLog | 8 ++++++++ INSTALL | 4 ++-- NEWS | 4 ++++ configure | 2 +- configure.ac | 2 +- manual/install.texi | 4 ++-- 6 files changed, 18 insertions(+), 6 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 3b61f167d2..b3d39a23fd 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,11 @@ +2017-06-19 Joseph Myers + + * configure.ac (libc_cv_compiler_ok): Require GCC 4.9 or later. + * configure: Regenerated. + * manual/install.texi (Tools for Compilation): Document + requirement for GCC 4.9 or later. + * INSTALL: Regenerated. + 2017-06-19 Florian Weimer [BZ #21624] diff --git a/INSTALL b/INSTALL index a2f5a40b17..71af35b1eb 100644 --- a/INSTALL +++ b/INSTALL @@ -420,9 +420,9 @@ build the GNU C Library: recommend GNU 'make' version 3.79. All earlier versions have severe bugs or lack features. - * GCC 4.7 or newer + * GCC 4.9 or newer - GCC 4.7 or higher is required. In general it is recommended to use + GCC 4.9 or higher is required. In general it is recommended to use the newest version of the compiler that is known to work for building the GNU C Library, as newer compilers usually produce better code. As of release time, GCC 6.3 is the newest compiler diff --git a/NEWS b/NEWS index 9ee22185e8..f81d02f1cb 100644 --- a/NEWS +++ b/NEWS @@ -107,6 +107,10 @@ Version 2.26 PTRACE_DISABLE_TE and PTRACE_TE_ABORT_RAND were added as those are supported by the s390 kernel. +* The minimum GCC version that can be used to build this version of the GNU + C Library is GCC 4.9. Older GCC versions, and non-GNU compilers, can + still be used to compile programs using the GNU C Library. + Security related changes: * The DNS stub resolver limits the advertised UDP buffer size to 1200 bytes, diff --git a/configure b/configure index 8390f2b86e..97a2dadf2d 100755 --- a/configure +++ b/configure @@ -5058,7 +5058,7 @@ int main () { -#if !defined __GNUC__ || __GNUC__ < 4 || (__GNUC__ == 4 && __GNUC_MINOR__ < 7) +#if !defined __GNUC__ || __GNUC__ < 4 || (__GNUC__ == 4 && __GNUC_MINOR__ < 9) #error insufficient compiler #endif ; diff --git a/configure.ac b/configure.ac index 4c0a18edd9..16e97d3930 100644 --- a/configure.ac +++ b/configure.ac @@ -1105,7 +1105,7 @@ AC_CHECK_PROG_VER(AWK, gawk, --version, AC_CACHE_CHECK([if $CC is sufficient to build libc], libc_cv_compiler_ok, [ AC_TRY_COMPILE([], [ -#if !defined __GNUC__ || __GNUC__ < 4 || (__GNUC__ == 4 && __GNUC_MINOR__ < 7) +#if !defined __GNUC__ || __GNUC__ < 4 || (__GNUC__ == 4 && __GNUC_MINOR__ < 9) #error insufficient compiler #endif], [libc_cv_compiler_ok=yes], diff --git a/manual/install.texi b/manual/install.texi index cbc912817a..65174a8b2e 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -467,9 +467,9 @@ recommend GNU @code{make} version 3.79. All earlier versions have severe bugs or lack features. @item -GCC 4.7 or newer +GCC 4.9 or newer -GCC 4.7 or higher is required. In general it is recommended to use +GCC 4.9 or higher is required. In general it is recommended to use the newest version of the compiler that is known to work for building @theglibc{}, as newer compilers usually produce better code. As of release time, GCC 6.3 is the newest compiler verified to work to build -- cgit v1.2.3 From 4efe3ce400d763b6c8186dd389371f6359b05c54 Mon Sep 17 00:00:00 2001 From: "Gabriel F. T. Gomes" Date: Tue, 11 Oct 2016 13:50:19 -0300 Subject: powerpc64le: Check for compiler features for float128 On powerpc64le, support for float128 will be enabled, which requires some compiler features to be present. This patch adds a configure test to check for such features, which are provided for powerpc64le since GCC 6.2. Tested for powerpc64 and powerpc64le. * INSTALL: Regenerate. * manual/install.texi (Recommended Tools for Compilation): Mention the powerpc64le-specific requirement in the manual. * sysdeps/powerpc/powerpc64le/configure.ac: New file with checks for the compiler features required for building float128. * sysdeps/powerpc/powerpc64le/configure: New, auto-generated file. --- ChangeLog | 10 +++++++ INSTALL | 5 ++++ manual/install.texi | 4 +++ sysdeps/powerpc/powerpc64le/configure | 45 ++++++++++++++++++++++++++++++++ sysdeps/powerpc/powerpc64le/configure.ac | 32 +++++++++++++++++++++++ 5 files changed, 96 insertions(+) create mode 100644 sysdeps/powerpc/powerpc64le/configure create mode 100644 sysdeps/powerpc/powerpc64le/configure.ac (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 9e19f62d9d..e0968afc8b 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,13 @@ +2017-06-26 Gabriel F. T. Gomes + Tulio Magno Quites Machado Filho + + * INSTALL: Regenerate. + * manual/install.texi (Recommended Tools for Compilation): Mention + the powerpc64le-specific requirement in the manual. + * sysdeps/powerpc/powerpc64le/configure.ac: New file with checks + for the compiler features required for building float128. + * sysdeps/powerpc/powerpc64le/configure: New, auto-generated file. + 2017-06-26 Tulio Magno Quites Machado Filho * malloc/tst-alloc_buffer.c (test_misaligned): Cast to char diff --git a/INSTALL b/INSTALL index 71af35b1eb..4e05e9922b 100644 --- a/INSTALL +++ b/INSTALL @@ -428,6 +428,11 @@ build the GNU C Library: better code. As of release time, GCC 6.3 is the newest compiler verified to work to build the GNU C Library. + For PowerPC 64-bits little-endian (powerpc64le), GCC 6.2 or higher + is required. This compiler version is the first to provide the + features required for building the GNU C Library with support for + '_Float128'. + For multi-arch support it is recommended to use a GCC which has been built with support for GNU indirect functions. This ensures that correct debugging information is generated for functions diff --git a/manual/install.texi b/manual/install.texi index 65174a8b2e..ff5661a4fd 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -475,6 +475,10 @@ the newest version of the compiler that is known to work for building release time, GCC 6.3 is the newest compiler verified to work to build @theglibc{}. +For PowerPC 64-bits little-endian (powerpc64le), GCC 6.2 or higher is +required. This compiler version is the first to provide the features +required for building @theglibc{} with support for @code{_Float128}. + For multi-arch support it is recommended to use a GCC which has been built with support for GNU indirect functions. This ensures that correct debugging information is generated for functions selected by IFUNC resolvers. This diff --git a/sysdeps/powerpc/powerpc64le/configure b/sysdeps/powerpc/powerpc64le/configure new file mode 100644 index 0000000000..78208d2eda --- /dev/null +++ b/sysdeps/powerpc/powerpc64le/configure @@ -0,0 +1,45 @@ +# This file is generated from configure.ac by Autoconf. DO NOT EDIT! + # Local configure fragment for sysdeps/powerpc/powerpc64le. + + +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking if $CC supports binary128 floating point type" >&5 +$as_echo_n "checking if $CC supports binary128 floating point type... " >&6; } +if ${libc_cv_compiler_powerpc64le_binary128_ok+:} false; then : + $as_echo_n "(cached) " >&6 +else + save_CFLAGS="$CFLAGS" +CFLAGS="$CFLAGS -Werror -mfloat128" +cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +__float128 a, b, c, d, e; +int i; + +__float128 +foobar (__float128 x) +{ + a = __builtin_nansq ("0"); + b = __builtin_huge_valq (); + c = __builtin_infq (); + d = __builtin_fabsq (x); + e = __builtin_nanq ("0"); + i = __builtin_signbit (x); + return __builtin_copysignq (x, x); +} + +_ACEOF +if ac_fn_c_try_compile "$LINENO"; then : + libc_cv_compiler_powerpc64le_binary128_ok=yes +else + libc_cv_compiler_powerpc64le_binary128_ok=no +fi +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext +CFLAGS="$save_CFLAGS" +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_compiler_powerpc64le_binary128_ok" >&5 +$as_echo "$libc_cv_compiler_powerpc64le_binary128_ok" >&6; } +if test "$libc_cv_compiler_powerpc64le_binary128_ok" != "yes"; then : + critic_missing="$critic_missing binary128 floating point type (GCC >= 6.2) is required on powerpc64le." +fi + +test -n "$critic_missing" && as_fn_error $? "*** $critic_missing" "$LINENO" 5 diff --git a/sysdeps/powerpc/powerpc64le/configure.ac b/sysdeps/powerpc/powerpc64le/configure.ac new file mode 100644 index 0000000000..e88e224ae1 --- /dev/null +++ b/sysdeps/powerpc/powerpc64le/configure.ac @@ -0,0 +1,32 @@ +GLIBC_PROVIDES dnl See aclocal.m4 in the top level source directory. +# Local configure fragment for sysdeps/powerpc/powerpc64le. + +dnl Require binary128 floating point support on powerpc64le (available in +dnl GCC 6.2). +AC_CACHE_CHECK([if $CC supports binary128 floating point type], + libc_cv_compiler_powerpc64le_binary128_ok, [dnl +save_CFLAGS="$CFLAGS" +CFLAGS="$CFLAGS -Werror -mfloat128" +AC_COMPILE_IFELSE([AC_LANG_SOURCE([[ +__float128 a, b, c, d, e; +int i; + +__float128 +foobar (__float128 x) +{ + a = __builtin_nansq ("0"); + b = __builtin_huge_valq (); + c = __builtin_infq (); + d = __builtin_fabsq (x); + e = __builtin_nanq ("0"); + i = __builtin_signbit (x); + return __builtin_copysignq (x, x); +} +]])], + [libc_cv_compiler_powerpc64le_binary128_ok=yes], + [libc_cv_compiler_powerpc64le_binary128_ok=no]) +CFLAGS="$save_CFLAGS"]) +AS_IF([test "$libc_cv_compiler_powerpc64le_binary128_ok" != "yes"], + [critic_missing="$critic_missing binary128 floating point type (GCC >= 6.2) is required on powerpc64le."]) + +test -n "$critic_missing" && AC_MSG_ERROR([*** $critic_missing]) -- cgit v1.2.3 From 073e8fa7739ed453d6854b834f290c263a6cdb9f Mon Sep 17 00:00:00 2001 From: Joseph Myers Date: Wed, 28 Jun 2017 11:31:50 +0000 Subject: Require binutils 2.25 or later to build glibc. This patch implements a requirement of binutils >= 2.25 (up from 2.22) to build glibc. Tests for 2.24 or later on x86_64 and s390 are removed. It was already the case, as indicated by buildbot results, that 2.24 was too old for building tests for 32-bit x86 (produced internal linker errors linking elf/tst-gnu2-tls1mod.so). I don't know if any configure tests for binutils features are obsolete given the increased version requirement. Tested for x86_64. * configure.ac (AS): Require binutils 2.25 or later. (LD): Likewise. * configure: Regenerated. * sysdeps/s390/configure.ac (AS): Remove version check. * sysdeps/s390/configure: Regenerated. * sysdeps/x86_64/configure.ac (AS): Remove version check. * sysdeps/x86_64/configure: Regenerated. * manual/install.texi (Tools for Compilation): Document requirement for binutils 2.25 or later. * INSTALL: Regenerated. --- ChangeLog | 13 +++++++++ INSTALL | 2 +- NEWS | 3 +++ configure | 2 +- configure.ac | 2 +- manual/install.texi | 2 +- sysdeps/s390/configure | 65 --------------------------------------------- sysdeps/s390/configure.ac | 6 ----- sysdeps/x86_64/configure | 64 -------------------------------------------- sysdeps/x86_64/configure.ac | 6 ----- 10 files changed, 20 insertions(+), 145 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index dc781f94d3..a1558d5bc5 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,16 @@ +2017-06-28 Joseph Myers + + * configure.ac (AS): Require binutils 2.25 or later. + (LD): Likewise. + * configure: Regenerated. + * sysdeps/s390/configure.ac (AS): Remove version check. + * sysdeps/s390/configure: Regenerated. + * sysdeps/x86_64/configure.ac (AS): Remove version check. + * sysdeps/x86_64/configure: Regenerated. + * manual/install.texi (Tools for Compilation): Document + requirement for binutils 2.25 or later. + * INSTALL: Regenerated. + 2017-06-28 H.J. Lu [BZ #14995] diff --git a/INSTALL b/INSTALL index 4e05e9922b..5c745244a6 100644 --- a/INSTALL +++ b/INSTALL @@ -448,7 +448,7 @@ build the GNU C Library: Check the FAQ for any special compiler issues on particular platforms. - * GNU 'binutils' 2.22 or later + * GNU 'binutils' 2.25 or later You must use GNU 'binutils' (as and ld) to build the GNU C Library. No other assembler or linker has the necessary functionality at the diff --git a/NEWS b/NEWS index 22c43f5977..ba3e77b66a 100644 --- a/NEWS +++ b/NEWS @@ -129,6 +129,9 @@ Version 2.26 C Library is GCC 4.9. Older GCC versions, and non-GNU compilers, can still be used to compile programs using the GNU C Library. +* The minimum GNU Binutils version that can be used to build this version of + the GNU C Library is Binutils 2.25. + * Support is added, on powerpc64le, x86_64, x86 and ia64, for interfaces supporting the _Float128 type from ISO/IEC TS 18661-3:2015. Most of the interfaces are taken from TS 18661-3. The type-generic macros in diff --git a/configure b/configure index b55fc92c53..8e5c5bdde0 100755 --- a/configure +++ b/configure @@ -4780,7 +4780,7 @@ $as_echo_n "checking version of $LD... " >&6; } ac_prog_version=`$LD --version 2>&1 | sed -n 's/^.*GNU ld.* \([0-9][0-9]*\.[0-9.]*\).*$/\1/p'` case $ac_prog_version in '') ac_prog_version="v. ?.??, bad"; ac_verc_fail=yes;; - 2.1[0-9][0-9]*|2.2[2-9]*|2.[3-9][0-9]*|[3-9].*|[1-9][0-9]*) + 2.1[0-9][0-9]*|2.2[5-9]*|2.[3-9][0-9]*|[3-9].*|[1-9][0-9]*) ac_prog_version="$ac_prog_version, ok"; ac_verc_fail=no;; *) ac_prog_version="$ac_prog_version, bad"; ac_verc_fail=yes;; diff --git a/configure.ac b/configure.ac index 5f62ecb718..7e03019c4f 100644 --- a/configure.ac +++ b/configure.ac @@ -1086,7 +1086,7 @@ if test -n "`$LD --version | sed -n 's/^GNU \(gold\).*$/\1/p'`"; then else AC_CHECK_PROG_VER(LD, $LD, --version, [GNU ld.* \([0-9][0-9]*\.[0-9.]*\)], - [2.1[0-9][0-9]*|2.2[2-9]*|2.[3-9][0-9]*|[3-9].*|[1-9][0-9]*], + [2.1[0-9][0-9]*|2.2[5-9]*|2.[3-9][0-9]*|[3-9].*|[1-9][0-9]*], LD=: critic_missing="$critic_missing GNU ld") fi diff --git a/manual/install.texi b/manual/install.texi index ff5661a4fd..03eb2dd93b 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -493,7 +493,7 @@ You can use whatever compiler you like to compile programs that use Check the FAQ for any special compiler issues on particular platforms. @item -GNU @code{binutils} 2.22 or later +GNU @code{binutils} 2.25 or later You must use GNU @code{binutils} (as and ld) to build @theglibc{}. No other assembler or linker has the necessary functionality at the diff --git a/sysdeps/s390/configure b/sysdeps/s390/configure index 347ac28528..d4a4a3dcf8 100644 --- a/sysdeps/s390/configure +++ b/sysdeps/s390/configure @@ -4,71 +4,6 @@ $as_echo "#define PI_STATIC_AND_HIDDEN 1" >>confdefs.h -for ac_prog in $AS -do - # Extract the first word of "$ac_prog", so it can be a program name with args. -set dummy $ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_AS+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$AS"; then - ac_cv_prog_AS="$AS" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_AS="$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -AS=$ac_cv_prog_AS -if test -n "$AS"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $AS" >&5 -$as_echo "$AS" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$AS" && break -done - -if test -z "$AS"; then - ac_verc_fail=yes -else - # Found it, now check the version. - { $as_echo "$as_me:${as_lineno-$LINENO}: checking version of $AS" >&5 -$as_echo_n "checking version of $AS... " >&6; } - ac_prog_version=`$AS --version 2>&1 | sed -n 's/^.*GNU assembler.* \([0-9]*\.[0-9.]*\).*$/\1/p'` - case $ac_prog_version in - '') ac_prog_version="v. ?.??, bad"; ac_verc_fail=yes;; - 2.2[4-9]*|2.[3-9][0-9]*|[3-9].*|[1-9][0-9]*) - ac_prog_version="$ac_prog_version, ok"; ac_verc_fail=no;; - *) ac_prog_version="$ac_prog_version, bad"; ac_verc_fail=yes;; - - esac - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_prog_version" >&5 -$as_echo "$ac_prog_version" >&6; } -fi -if test $ac_verc_fail = yes; then - critic_missing="$critic_missing The program AS is required in version >= 2.24 for target S390." -fi - - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_tbegin" >&5 $as_echo_n "checking for __builtin_tbegin... " >&6; } if ${libc_cv_gcc_builtin_tbegin+:} false; then : diff --git a/sysdeps/s390/configure.ac b/sysdeps/s390/configure.ac index 8a782e7c3a..7d0b5ce46f 100644 --- a/sysdeps/s390/configure.ac +++ b/sysdeps/s390/configure.ac @@ -5,12 +5,6 @@ dnl It is always possible to access static and hidden symbols in an dnl position independent way. AC_DEFINE(PI_STATIC_AND_HIDDEN) -dnl Accept as 2.24 or newer. -AC_CHECK_PROG_VER(AS, $AS, --version, - [GNU assembler.* \([0-9]*\.[0-9.]*\)], - [2.2[4-9]*|2.[3-9][0-9]*|[3-9].*|[1-9][0-9]*], critic_missing="$critic_missing The program AS is required in version >= 2.24 for target S390.") - - AC_CACHE_CHECK(for __builtin_tbegin, libc_cv_gcc_builtin_tbegin, [dnl cat > conftest.c <<\EOF #include diff --git a/sysdeps/x86_64/configure b/sysdeps/x86_64/configure index 2d14c344df..efef46b1b7 100644 --- a/sysdeps/x86_64/configure +++ b/sysdeps/x86_64/configure @@ -1,70 +1,6 @@ # This file is generated from configure.ac by Autoconf. DO NOT EDIT! # Local configure fragment for sysdeps/x86_64. -for ac_prog in $AS -do - # Extract the first word of "$ac_prog", so it can be a program name with args. -set dummy $ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_AS+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$AS"; then - ac_cv_prog_AS="$AS" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_AS="$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -AS=$ac_cv_prog_AS -if test -n "$AS"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $AS" >&5 -$as_echo "$AS" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$AS" && break -done - -if test -z "$AS"; then - ac_verc_fail=yes -else - # Found it, now check the version. - { $as_echo "$as_me:${as_lineno-$LINENO}: checking version of $AS" >&5 -$as_echo_n "checking version of $AS... " >&6; } - ac_prog_version=`$AS --version 2>&1 | sed -n 's/^.*GNU assembler.* \([0-9]*\.[0-9.]*\).*$/\1/p'` - case $ac_prog_version in - '') ac_prog_version="v. ?.??, bad"; ac_verc_fail=yes;; - 2.2[4-9]*|2.[3-9][0-9]*|[3-9].*|[1-9][0-9]*) - ac_prog_version="$ac_prog_version, ok"; ac_verc_fail=no;; - *) ac_prog_version="$ac_prog_version, bad"; ac_verc_fail=yes;; - - esac - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_prog_version" >&5 -$as_echo "$ac_prog_version" >&6; } -fi -if test $ac_verc_fail = yes; then - critic_missing="$critic_missing The program AS is required in version >= 2.24 for target x86_64." -fi - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for AVX512DQ support in assembler" >&5 $as_echo_n "checking for AVX512DQ support in assembler... " >&6; } if ${libc_cv_asm_avx512dq+:} false; then : diff --git a/sysdeps/x86_64/configure.ac b/sysdeps/x86_64/configure.ac index 7d8aaafc0c..fa86e953ee 100644 --- a/sysdeps/x86_64/configure.ac +++ b/sysdeps/x86_64/configure.ac @@ -1,12 +1,6 @@ GLIBC_PROVIDES dnl See aclocal.m4 in the top level source directory. # Local configure fragment for sysdeps/x86_64. -dnl Accept as 2.24 or newer for AVX512 load and store. -AC_CHECK_PROG_VER(AS, $AS, --version, - [GNU assembler.* \([0-9]*\.[0-9.]*\)], - [2.2[4-9]*|2.[3-9][0-9]*|[3-9].*|[1-9][0-9]*], - critic_missing="$critic_missing The program AS is required in version >= 2.24 for target x86_64.") - dnl Check if asm supports AVX512DQ. AC_CACHE_CHECK(for AVX512DQ support in assembler, libc_cv_asm_avx512dq, [dnl cat > conftest.s <<\EOF -- cgit v1.2.3 From d5c3fafc4307c9b7a4c7d5cb381fcdbfad340bcc Mon Sep 17 00:00:00 2001 From: DJ Delorie Date: Thu, 6 Jul 2017 13:37:30 -0400 Subject: Add per-thread cache to malloc * config.make.in: Enable experimental malloc option. * configure.ac: Likewise. * configure: Regenerate. * manual/install.texi: Document it. * INSTALL: Regenerate. * malloc/Makefile: Likewise. * malloc/malloc.c: Add per-thread cache (tcache). (tcache_put): New. (tcache_get): New. (tcache_thread_freeres): New. (tcache_init): New. (__libc_malloc): Use cached chunks if available. (__libc_free): Initialize tcache if needed. (__libc_realloc): Likewise. (__libc_calloc): Likewise. (_int_malloc): Prefill tcache when appropriate. (_int_free): Likewise. (do_set_tcache_max): New. (do_set_tcache_count): New. (do_set_tcache_unsorted_limit): New. * manual/probes.texi: Document new probes. * malloc/arena.c: Add new tcache tunables. * elf/dl-tunables.list: Likewise. * manual/tunables.texi: Document them. * NEWS: Mention the per-thread cache. --- ChangeLog | 28 +++++ INSTALL | 6 + NEWS | 8 ++ config.make.in | 2 + configure | 13 ++ configure.ac | 7 ++ elf/dl-tunables.list | 12 ++ malloc/Makefile | 5 + malloc/arena.c | 11 ++ malloc/malloc.c | 350 +++++++++++++++++++++++++++++++++++++++++++++++++-- manual/install.texi | 6 + manual/probes.texi | 19 +++ manual/tunables.texi | 32 +++++ 13 files changed, 490 insertions(+), 9 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index ac598984e3..1c51b3be29 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,31 @@ +2017-07-06 DJ Delorie + + * config.make.in: Enable experimental malloc option. + * configure.ac: Likewise. + * configure: Regenerate. + * manual/install.texi: Document it. + * INSTALL: Regenerate. + * malloc/Makefile: Likewise. + * malloc/malloc.c: Add per-thread cache (tcache). + (tcache_put): New. + (tcache_get): New. + (tcache_thread_freeres): New. + (tcache_init): New. + (__libc_malloc): Use cached chunks if available. + (__libc_free): Initialize tcache if needed. + (__libc_realloc): Likewise. + (__libc_calloc): Likewise. + (_int_malloc): Prefill tcache when appropriate. + (_int_free): Likewise. + (do_set_tcache_max): New. + (do_set_tcache_count): New. + (do_set_tcache_unsorted_limit): New. + * manual/probes.texi: Document new probes. + * malloc/arena.c: Add new tcache tunables. + * elf/dl-tunables.list: Likewise. + * manual/tunables.texi: Document them. + * NEWS: Mention the per-thread cache. + 2017-07-06 Joseph Myers * iconvdata/tst-loading.c (TIMEOUT): Define to 30. diff --git a/INSTALL b/INSTALL index 5c745244a6..0ff87ed603 100644 --- a/INSTALL +++ b/INSTALL @@ -200,6 +200,12 @@ will be used, and CFLAGS sets optimization options for the compiler. libnss_nisplus are not built at all. Use this option to enable libnsl with all depending NSS modules and header files. +'--disable-experimental-malloc' + By default, a per-thread cache is enabled in 'malloc'. While this + cache can be disabled on a per-application basis using tunables + (set glibc.malloc.tcache_count to zero), this option can be used to + remove it from the build completely. + '--build=BUILD-SYSTEM' '--host=HOST-SYSTEM' These options are for cross-compiling. If you specify both options diff --git a/NEWS b/NEWS index 616b3244e7..1266276fcd 100644 --- a/NEWS +++ b/NEWS @@ -9,6 +9,14 @@ Version 2.26 Major new features: +* A per-thread cache has been added to malloc. Access to the cache requires + no locks and therefore significantly accelerates the fast path to allocate + and free small amounts of memory. Refilling an empty cache requires locking + the underlying arena. Performance measurements show significant gains in a + wide variety of user workloads. Workloads were captured using a special + instrumented malloc and analyzed with a malloc simulator. Contributed by + DJ Delorie with the help of Florian Weimer, and Carlos O'Donell. + * Unicode 10.0.0 Support: Character encoding, character type info, and transliteration tables are all updated to Unicode 10.0.0, using generator scripts contributed by Mike FABIAN (Red Hat). diff --git a/config.make.in b/config.make.in index dadabf9b6a..5a4a054612 100644 --- a/config.make.in +++ b/config.make.in @@ -78,6 +78,8 @@ multi-arch = @multi_arch@ mach-interface-list = @mach_interface_list@ +experimental-malloc = @experimental_malloc@ + nss-crypt = @libc_cv_nss_crypt@ static-nss-crypt = @libc_cv_static_nss_crypt@ diff --git a/configure b/configure index 8e5c5bdde0..d8e1c50e11 100755 --- a/configure +++ b/configure @@ -674,6 +674,7 @@ build_obsolete_nsl link_obsolete_rpc libc_cv_static_nss_crypt libc_cv_nss_crypt +experimental_malloc enable_werror all_warnings force_install @@ -779,6 +780,7 @@ enable_kernel enable_all_warnings enable_werror enable_multi_arch +enable_experimental_malloc enable_nss_crypt enable_obsolete_rpc enable_obsolete_nsl @@ -1450,6 +1452,8 @@ Optional Features: --disable-werror do not build with -Werror --enable-multi-arch enable single DSO with optimizations for multiple architectures + --disable-experimental-malloc + disable experimental malloc features --enable-nss-crypt enable libcrypt to use nss --enable-obsolete-rpc build and install the obsolete RPC code for link-time usage @@ -3522,6 +3526,15 @@ else fi +# Check whether --enable-experimental-malloc was given. +if test "${enable_experimental_malloc+set}" = set; then : + enableval=$enable_experimental_malloc; experimental_malloc=$enableval +else + experimental_malloc=yes +fi + + + # Check whether --enable-nss-crypt was given. if test "${enable_nss_crypt+set}" = set; then : enableval=$enable_nss_crypt; nss_crypt=$enableval diff --git a/configure.ac b/configure.ac index 7e03019c4f..77456aa8d9 100644 --- a/configure.ac +++ b/configure.ac @@ -313,6 +313,13 @@ AC_ARG_ENABLE([multi-arch], [multi_arch=$enableval], [multi_arch=default]) +AC_ARG_ENABLE([experimental-malloc], + AC_HELP_STRING([--disable-experimental-malloc], + [disable experimental malloc features]), + [experimental_malloc=$enableval], + [experimental_malloc=yes]) +AC_SUBST(experimental_malloc) + AC_ARG_ENABLE([nss-crypt], AC_HELP_STRING([--enable-nss-crypt], [enable libcrypt to use nss]), diff --git a/elf/dl-tunables.list b/elf/dl-tunables.list index df4f9622b4..c188c6ad52 100644 --- a/elf/dl-tunables.list +++ b/elf/dl-tunables.list @@ -76,6 +76,18 @@ glibc { minval: 1 security_level: SXID_IGNORE } + tcache_max { + type: SIZE_T + security_level: SXID_ERASE + } + tcache_count { + type: SIZE_T + security_level: SXID_ERASE + } + tcache_unsorted_limit { + type: SIZE_T + security_level: SXID_ERASE + } } tune { hwcap_mask { diff --git a/malloc/Makefile b/malloc/Makefile index b50de7cd6c..3fa395b949 100644 --- a/malloc/Makefile +++ b/malloc/Makefile @@ -189,6 +189,11 @@ tst-malloc-usable-static-ENV = $(tst-malloc-usable-ENV) tst-malloc-usable-tunables-ENV = GLIBC_TUNABLES=glibc.malloc.check=3 tst-malloc-usable-static-tunables-ENV = $(tst-malloc-usable-tunables-ENV) +ifeq ($(experimental-malloc),yes) +CPPFLAGS-malloc.c += -DUSE_TCACHE=1 +else +CPPFLAGS-malloc.c += -DUSE_TCACHE=0 +endif # Uncomment this for test releases. For public releases it is too expensive. #CPPFLAGS-malloc.o += -DMALLOC_DEBUG=1 diff --git a/malloc/arena.c b/malloc/arena.c index 660d638c93..dc14fae152 100644 --- a/malloc/arena.c +++ b/malloc/arena.c @@ -236,6 +236,11 @@ TUNABLE_CALLBACK_FNDECL (set_perturb_byte, int32_t) TUNABLE_CALLBACK_FNDECL (set_trim_threshold, size_t) TUNABLE_CALLBACK_FNDECL (set_arena_max, size_t) TUNABLE_CALLBACK_FNDECL (set_arena_test, size_t) +#if USE_TCACHE +TUNABLE_CALLBACK_FNDECL (set_tcache_max, size_t) +TUNABLE_CALLBACK_FNDECL (set_tcache_count, size_t) +TUNABLE_CALLBACK_FNDECL (set_tcache_unsorted_limit, size_t) +#endif #else /* Initialization routine. */ #include @@ -322,6 +327,12 @@ ptmalloc_init (void) TUNABLE_GET (mmap_max, int32_t, TUNABLE_CALLBACK (set_mmaps_max)); TUNABLE_GET (arena_max, size_t, TUNABLE_CALLBACK (set_arena_max)); TUNABLE_GET (arena_test, size_t, TUNABLE_CALLBACK (set_arena_test)); +#if USE_TCACHE + TUNABLE_GET (tcache_max, size_t, TUNABLE_CALLBACK (set_tcache_max)); + TUNABLE_GET (tcache_count, size_t, TUNABLE_CALLBACK (set_tcache_count)); + TUNABLE_GET (tcache_unsorted_limit, size_t, + TUNABLE_CALLBACK (set_tcache_unsorted_limit)); +#endif __libc_lock_unlock (main_arena.mutex); #else const char *s = NULL; diff --git a/malloc/malloc.c b/malloc/malloc.c index aa45626093..2527e25047 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -238,6 +238,9 @@ /* For ALIGN_UP et. al. */ #include +/* For DIAG_PUSH/POP_NEEDS_COMMENT et al. */ +#include + #include /* @@ -296,6 +299,30 @@ __malloc_assert (const char *assertion, const char *file, unsigned int line, } #endif +#if USE_TCACHE +/* We want 64 entries. This is an arbitrary limit, which tunables can reduce. */ +# define TCACHE_MAX_BINS 64 +# define MAX_TCACHE_SIZE tidx2usize (TCACHE_MAX_BINS-1) + +/* Only used to pre-fill the tunables. */ +# define tidx2usize(idx) (((size_t) idx) * MALLOC_ALIGNMENT + MINSIZE - SIZE_SZ) + +/* When "x" is from chunksize(). */ +# define csize2tidx(x) (((x) - MINSIZE + MALLOC_ALIGNMENT - 1) / MALLOC_ALIGNMENT) +/* When "x" is a user-provided size. */ +# define usize2tidx(x) csize2tidx (request2size (x)) + +/* With rounding and alignment, the bins are... + idx 0 bytes 0..24 (64-bit) or 0..12 (32-bit) + idx 1 bytes 25..40 or 13..20 + idx 2 bytes 41..56 or 21..28 + etc. */ + +/* This is another arbitrary limit, which tunables can change. Each + tcache bin will hold at most this number of chunks. */ +# define TCACHE_FILL_COUNT 7 +#endif + /* REALLOC_ZERO_BYTES_FREES should be set if a call to @@ -1712,6 +1739,17 @@ struct malloc_par /* First address handed out by MORECORE/sbrk. */ char *sbrk_base; + +#if USE_TCACHE + /* Maximum number of buckets to use. */ + size_t tcache_bins; + size_t tcache_max_bytes; + /* Maximum number of chunks in each bucket. */ + size_t tcache_count; + /* Maximum number of chunks to remove from the unsorted list, which + aren't used to prefill the cache. */ + size_t tcache_unsorted_limit; +#endif }; /* There are several instances of this struct ("arenas") in this @@ -1750,6 +1788,13 @@ static struct malloc_par mp_ = .trim_threshold = DEFAULT_TRIM_THRESHOLD, #define NARENAS_FROM_NCORES(n) ((n) * (sizeof (long) == 4 ? 2 : 8)) .arena_test = NARENAS_FROM_NCORES (1) +#if USE_TCACHE + , + .tcache_count = TCACHE_FILL_COUNT, + .tcache_bins = TCACHE_MAX_BINS, + .tcache_max_bytes = tidx2usize (TCACHE_MAX_BINS-1), + .tcache_unsorted_limit = 0 /* No limit. */ +#endif }; /* Maximum size of memory handled in fastbins. */ @@ -2875,6 +2920,124 @@ mremap_chunk (mchunkptr p, size_t new_size) /*------------------------ Public wrappers. --------------------------------*/ +#if USE_TCACHE + +/* We overlay this structure on the user-data portion of a chunk when + the chunk is stored in the per-thread cache. */ +typedef struct tcache_entry +{ + struct tcache_entry *next; +} tcache_entry; + +/* There is one of these for each thread, which contains the + per-thread cache (hence "tcache_perthread_struct"). Keeping + overall size low is mildly important. Note that COUNTS and ENTRIES + are redundant (we could have just counted the linked list each + time), this is for performance reasons. */ +typedef struct tcache_perthread_struct +{ + char counts[TCACHE_MAX_BINS]; + tcache_entry *entries[TCACHE_MAX_BINS]; +} tcache_perthread_struct; + +static __thread char tcache_shutting_down = 0; +static __thread tcache_perthread_struct *tcache = NULL; + +/* Caller must ensure that we know tc_idx is valid and there's room + for more chunks. */ +static void +tcache_put (mchunkptr chunk, size_t tc_idx) +{ + tcache_entry *e = (tcache_entry *) chunk2mem (chunk); + assert (tc_idx < TCACHE_MAX_BINS); + e->next = tcache->entries[tc_idx]; + tcache->entries[tc_idx] = e; + ++(tcache->counts[tc_idx]); +} + +/* Caller must ensure that we know tc_idx is valid and there's + available chunks to remove. */ +static void * +tcache_get (size_t tc_idx) +{ + tcache_entry *e = tcache->entries[tc_idx]; + assert (tc_idx < TCACHE_MAX_BINS); + assert (tcache->entries[tc_idx] > 0); + tcache->entries[tc_idx] = e->next; + --(tcache->counts[tc_idx]); + return (void *) e; +} + +static void __attribute__ ((section ("__libc_thread_freeres_fn"))) +tcache_thread_freeres (void) +{ + int i; + tcache_perthread_struct *tcache_tmp = tcache; + + if (!tcache) + return; + + tcache = NULL; + + for (i = 0; i < TCACHE_MAX_BINS; ++i) + { + while (tcache_tmp->entries[i]) + { + tcache_entry *e = tcache_tmp->entries[i]; + tcache_tmp->entries[i] = e->next; + __libc_free (e); + } + } + + __libc_free (tcache_tmp); + + tcache_shutting_down = 1; +} +text_set_element (__libc_thread_subfreeres, tcache_thread_freeres); + +static void +tcache_init(void) +{ + mstate ar_ptr; + void *victim = 0; + const size_t bytes = sizeof (tcache_perthread_struct); + + if (tcache_shutting_down) + return; + + arena_get (ar_ptr, bytes); + victim = _int_malloc (ar_ptr, bytes); + if (!victim && ar_ptr != NULL) + { + ar_ptr = arena_get_retry (ar_ptr, bytes); + victim = _int_malloc (ar_ptr, bytes); + } + + + if (ar_ptr != NULL) + __libc_lock_unlock (ar_ptr->mutex); + + /* In a low memory situation, we may not be able to allocate memory + - in which case, we just keep trying later. However, we + typically do this very early, so either there is sufficient + memory, or there isn't enough memory to do non-trivial + allocations anyway. */ + if (victim) + { + tcache = (tcache_perthread_struct *) victim; + memset (tcache, 0, sizeof (tcache_perthread_struct)); + } + +} + +#define MAYBE_INIT_TCACHE() \ + if (__glibc_unlikely (tcache == NULL)) \ + tcache_init(); + +#else +#define MAYBE_INIT_TCACHE() +#endif + void * __libc_malloc (size_t bytes) { @@ -2885,6 +3048,23 @@ __libc_malloc (size_t bytes) = atomic_forced_read (__malloc_hook); if (__builtin_expect (hook != NULL, 0)) return (*hook)(bytes, RETURN_ADDRESS (0)); +#if USE_TCACHE + /* int_free also calls request2size, be careful to not pad twice. */ + size_t tbytes = request2size (bytes); + size_t tc_idx = csize2tidx (tbytes); + + MAYBE_INIT_TCACHE (); + + DIAG_PUSH_NEEDS_COMMENT; + if (tc_idx < mp_.tcache_bins + /*&& tc_idx < TCACHE_MAX_BINS*/ /* to appease gcc */ + && tcache + && tcache->entries[tc_idx] != NULL) + { + return tcache_get (tc_idx); + } + DIAG_POP_NEEDS_COMMENT; +#endif arena_get (ar_ptr, bytes); @@ -2944,6 +3124,8 @@ __libc_free (void *mem) return; } + MAYBE_INIT_TCACHE (); + ar_ptr = arena_for_chunk (p); _int_free (ar_ptr, p, 0); } @@ -2981,7 +3163,10 @@ __libc_realloc (void *oldmem, size_t bytes) if (chunk_is_mmapped (oldp)) ar_ptr = NULL; else - ar_ptr = arena_for_chunk (oldp); + { + MAYBE_INIT_TCACHE (); + ar_ptr = arena_for_chunk (oldp); + } /* Little security check which won't hurt performance: the allocator never wrapps around at the end of the address space. Therefore @@ -3206,6 +3391,8 @@ __libc_calloc (size_t n, size_t elem_size) sz = bytes; + MAYBE_INIT_TCACHE (); + arena_get (av, sz); if (av) { @@ -3336,6 +3523,10 @@ _int_malloc (mstate av, size_t bytes) mchunkptr fwd; /* misc temp for linking */ mchunkptr bck; /* misc temp for linking */ +#if USE_TCACHE + size_t tcache_unsorted_count; /* count of unsorted chunks processed */ +#endif + const char *errstr = NULL; /* @@ -3365,19 +3556,22 @@ _int_malloc (mstate av, size_t bytes) can try it without checking, which saves some time on this fast path. */ +#define REMOVE_FB(fb, victim, pp) \ + do \ + { \ + victim = pp; \ + if (victim == NULL) \ + break; \ + } \ + while ((pp = catomic_compare_and_exchange_val_acq (fb, victim->fd, victim)) \ + != victim); \ + if ((unsigned long) (nb) <= (unsigned long) (get_max_fast ())) { idx = fastbin_index (nb); mfastbinptr *fb = &fastbin (av, idx); mchunkptr pp = *fb; - do - { - victim = pp; - if (victim == NULL) - break; - } - while ((pp = catomic_compare_and_exchange_val_acq (fb, victim->fd, victim)) - != victim); + REMOVE_FB (fb, victim, pp); if (victim != 0) { if (__builtin_expect (fastbin_index (chunksize (victim)) != idx, 0)) @@ -3388,6 +3582,26 @@ _int_malloc (mstate av, size_t bytes) return NULL; } check_remalloced_chunk (av, victim, nb); +#if USE_TCACHE + /* While we're here, if we see other chunks of the same size, + stash them in the tcache. */ + size_t tc_idx = csize2tidx (nb); + if (tcache && tc_idx < mp_.tcache_bins) + { + mchunkptr tc_victim; + + /* While bin not empty and tcache not full, copy chunks over. */ + while (tcache->counts[tc_idx] < mp_.tcache_count + && (pp = *fb) != NULL) + { + REMOVE_FB (fb, tc_victim, pp); + if (tc_victim != 0) + { + tcache_put (tc_victim, tc_idx); + } + } + } +#endif void *p = chunk2mem (victim); alloc_perturb (p, bytes); return p; @@ -3426,6 +3640,32 @@ _int_malloc (mstate av, size_t bytes) if (av != &main_arena) set_non_main_arena (victim); check_malloced_chunk (av, victim, nb); +#if USE_TCACHE + /* While we're here, if we see other chunks of the same size, + stash them in the tcache. */ + size_t tc_idx = csize2tidx (nb); + if (tcache && tc_idx < mp_.tcache_bins) + { + mchunkptr tc_victim; + + /* While bin not empty and tcache not full, copy chunks over. */ + while (tcache->counts[tc_idx] < mp_.tcache_count + && (tc_victim = last (bin)) != bin) + { + if (tc_victim != 0) + { + bck = tc_victim->bk; + set_inuse_bit_at_offset (tc_victim, nb); + if (av != &main_arena) + set_non_main_arena (tc_victim); + bin->bk = bck; + bck->fd = bin; + + tcache_put (tc_victim, tc_idx); + } + } + } +#endif void *p = chunk2mem (victim); alloc_perturb (p, bytes); return p; @@ -3464,6 +3704,16 @@ _int_malloc (mstate av, size_t bytes) otherwise need to expand memory to service a "small" request. */ +#if USE_TCACHE + INTERNAL_SIZE_T tcache_nb = 0; + size_t tc_idx = csize2tidx (nb); + if (tcache && tc_idx < mp_.tcache_bins) + tcache_nb = nb; + int return_cached = 0; + + tcache_unsorted_count = 0; +#endif + for (;; ) { int iters = 0; @@ -3524,10 +3774,26 @@ _int_malloc (mstate av, size_t bytes) set_inuse_bit_at_offset (victim, size); if (av != &main_arena) set_non_main_arena (victim); +#if USE_TCACHE + /* Fill cache first, return to user only if cache fills. + We may return one of these chunks later. */ + if (tcache_nb + && tcache->counts[tc_idx] < mp_.tcache_count) + { + tcache_put (victim, tc_idx); + return_cached = 1; + continue; + } + else + { +#endif check_malloced_chunk (av, victim, nb); void *p = chunk2mem (victim); alloc_perturb (p, bytes); return p; +#if USE_TCACHE + } +#endif } /* place chunk in bin */ @@ -3594,11 +3860,31 @@ _int_malloc (mstate av, size_t bytes) fwd->bk = victim; bck->fd = victim; +#if USE_TCACHE + /* If we've processed as many chunks as we're allowed while + filling the cache, return one of the cached ones. */ + ++tcache_unsorted_count; + if (return_cached + && mp_.tcache_unsorted_limit > 0 + && tcache_unsorted_count > mp_.tcache_unsorted_limit) + { + return tcache_get (tc_idx); + } +#endif + #define MAX_ITERS 10000 if (++iters >= MAX_ITERS) break; } +#if USE_TCACHE + /* If all the small chunks we found ended up cached, return one now. */ + if (return_cached) + { + return tcache_get (tc_idx); + } +#endif + /* If a large request, scan through the chunks of current bin in sorted order to find smallest that fits. Use the skip list for this. @@ -3884,6 +4170,20 @@ _int_free (mstate av, mchunkptr p, int have_lock) check_inuse_chunk(av, p); +#if USE_TCACHE + { + size_t tc_idx = csize2tidx (size); + + if (tcache + && tc_idx < mp_.tcache_bins + && tcache->counts[tc_idx] < mp_.tcache_count) + { + tcache_put (p, tc_idx); + return; + } + } +#endif + /* If eligible, place chunk on a fastbin so it can be found and used quickly in malloc. @@ -4845,6 +5145,38 @@ do_set_arena_max (size_t value) return 1; } +#if USE_TCACHE +static inline int +__always_inline +do_set_tcache_max (size_t value) +{ + if (value >= 0 && value <= MAX_TCACHE_SIZE) + { + LIBC_PROBE (memory_tunable_tcache_max_bytes, 2, value, mp_.tcache_max_bytes); + mp_.tcache_max_bytes = value; + mp_.tcache_bins = csize2tidx (request2size(value)) + 1; + } + return 1; +} + +static inline int +__always_inline +do_set_tcache_count (size_t value) +{ + LIBC_PROBE (memory_tunable_tcache_count, 2, value, mp_.tcache_count); + mp_.tcache_count = value; + return 1; +} + +static inline int +__always_inline +do_set_tcache_unsorted_limit (size_t value) +{ + LIBC_PROBE (memory_tunable_tcache_unsorted_limit, 2, value, mp_.tcache_unsorted_limit); + mp_.tcache_unsorted_limit = value; + return 1; +} +#endif int __libc_mallopt (int param_number, int value) diff --git a/manual/install.texi b/manual/install.texi index 03eb2dd93b..b8deb9ceba 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -232,6 +232,12 @@ libnss_nisplus are not built at all. Use this option to enable libnsl with all depending NSS modules and header files. +@item --disable-experimental-malloc +By default, a per-thread cache is enabled in @code{malloc}. While +this cache can be disabled on a per-application basis using tunables +(set glibc.malloc.tcache_count to zero), this option can be used to +remove it from the build completely. + @item --build=@var{build-system} @itemx --host=@var{host-system} These options are for cross-compiling. If you specify both options and diff --git a/manual/probes.texi b/manual/probes.texi index eb91c62703..96acaed206 100644 --- a/manual/probes.texi +++ b/manual/probes.texi @@ -231,6 +231,25 @@ dynamic brk/mmap thresholds. Argument @var{$arg1} and @var{$arg2} are the adjusted mmap and trim thresholds, respectively. @end deftp +@deftp Probe memory_tunable_tcache_max_bytes (int @var{$arg1}, int @var{$arg2}) +This probe is triggered when the @code{glibc.malloc.tcache_max} +tunable is set. Argument @var{$arg1} is the requested value, and +@var{$arg2} is the previous value of this tunable. +@end deftp + +@deftp Probe memory_tunable_tcache_count (int @var{$arg1}, int @var{$arg2}) +This probe is triggered when the @code{glibc.malloc.tcache_count} +tunable is set. Argument @var{$arg1} is the requested value, and +@var{$arg2} is the previous value of this tunable. +@end deftp + +@deftp Probe memory_tunable_tcache_unsorted_limit (int @var{$arg1}, int @var{$arg2}) +This probe is triggered when the +@code{glibc.malloc.tcache_unsorted_limit} tunable is set. Argument +@var{$arg1} is the requested value, and @var{$arg2} is the previous +value of this tunable. +@end deftp + @node Mathematical Function Probes @section Mathematical Function Probes diff --git a/manual/tunables.texi b/manual/tunables.texi index 9331b03702..b16d591b90 100644 --- a/manual/tunables.texi +++ b/manual/tunables.texi @@ -193,6 +193,38 @@ systems the limit is twice the number of cores online and on 64-bit systems, it is 8 times the number of cores online. @end deftp +@deftp Tunable glibc.malloc.tcache_max +The maximum size of a request (in bytes) which may be met via the +per-thread cache. The default (and maximum) value is 1032 bytes on +64-bit systems and 516 bytes on 32-bit systems. +@end deftp + +@deftp Tunable glibc.malloc.tcache_count +The maximum number of chunks of each size to cache. The default is 7. +There is no upper limit, other than available system memory. If set +to zero, the per-thread cache is effectively disabled. + +The approximate maximum overhead of the per-thread cache is thus equal +to the number of bins times the chunk count in each bin times the size +of each chunk. With defaults, the approximate maximum overhead of the +per-thread cache is approximately 236 KB on 64-bit systems and 118 KB +on 32-bit systems. +@end deftp + +@deftp Tunable glibc.malloc.tcache_unsorted_limit +When the user requests memory and the request cannot be met via the +per-thread cache, the arenas are used to meet the request. At this +time, additional chunks will be moved from existing arena lists to +pre-fill the corresponding cache. While copies from the fastbins, +smallbins, and regular bins are bounded and predictable due to the bin +sizes, copies from the unsorted bin are not bounded, and incur +additional time penalties as they need to be sorted as they're +scanned. To make scanning the unsorted list more predictable and +bounded, the user may set this tunable to limit the number of chunks +that are scanned from the unsorted list while searching for chunks to +pre-fill the per-thread cache with. The default, or when set to zero, +is no limit. + @node Hardware Capability Tunables @section Hardware Capability Tunables @cindex hardware capability tunables -- cgit v1.2.3 From 15192aaa252afddb97413812e32eaea80c55af19 Mon Sep 17 00:00:00 2001 From: Siddhesh Poyarekar Date: Wed, 2 Aug 2017 18:22:58 +0530 Subject: Update contributors and latest gcc and binutils versions --- ChangeLog | 5 +++++ INSTALL | 4 ++-- manual/contrib.texi | 28 ++++++++++++++++++++++++++++ manual/install.texi | 4 ++-- 4 files changed, 37 insertions(+), 4 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 3c3279c92c..98610b73ac 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,10 @@ 2017-08-02 Siddhesh Poyarekar + * manual/contrib.texi: Update contributors. + * manual/install.texi: Update latest tested versions of gcc and + binutils. + * INSTALL: Regenerate. + * NEWS: Add bugs fixed. * po/fr.po: Update translations. diff --git a/INSTALL b/INSTALL index 0ff87ed603..4d9024c667 100644 --- a/INSTALL +++ b/INSTALL @@ -431,7 +431,7 @@ build the GNU C Library: GCC 4.9 or higher is required. In general it is recommended to use the newest version of the compiler that is known to work for building the GNU C Library, as newer compilers usually produce - better code. As of release time, GCC 6.3 is the newest compiler + better code. As of release time, GCC 7.1 is the newest compiler verified to work to build the GNU C Library. For PowerPC 64-bits little-endian (powerpc64le), GCC 6.2 or higher @@ -458,7 +458,7 @@ build the GNU C Library: You must use GNU 'binutils' (as and ld) to build the GNU C Library. No other assembler or linker has the necessary functionality at the - moment. As of release time, GNU 'binutils' 2.25 is the newest + moment. As of release time, GNU 'binutils' 2.27 is the newest verified to work to build the GNU C Library. * GNU 'texinfo' 4.7 or later diff --git a/manual/contrib.texi b/manual/contrib.texi index cc52f60779..ffdd77191c 100644 --- a/manual/contrib.texi +++ b/manual/contrib.texi @@ -15,6 +15,9 @@ order. Nick Alcock for contributing fixes to allow @theglibc{} to be built with the stack smashing protector enabled. +@item +John David Anglin for various fixes to the hppa port. + @item Ryan S. Arnold for his improvements for Linux on PowerPC and his direction as FSF Project Steward for @theglibc{}. @@ -58,6 +61,9 @@ committee. @item Thomas Bushnell for his contributions to Hurd. +@item +DJ Delorie for various fixes. + @item Wilco Dijkstra for various fixes. @@ -102,6 +108,10 @@ part of @theglibc{} steering committee. @item Steve Ellcey for various fixes. +@item + +Mike FABIAN for various fixes to locales. + @item Tulio Magno Quites Machado Filho for adding a new class of installed headers for low-level platform-specific functionality and one such for @@ -195,6 +205,9 @@ Thorsten Kukuk for providing an implementation for NIS (YP) and NIS+, securelevel 0, 1 and 2 and for the implementation for a caching daemon for NSS (@file{nscd}). +@item +Akhilesh Kumar for various fixes to locales. + @item Jeff Law for various fixes. @@ -215,6 +228,9 @@ environment under x86-64 (x32), for porting to Linux on IA64, for improved string functions, a framework for testing IFUNC implementations, and many bug fixes. +@item +Rafal Luzynski for various fixes to locales. + @item Luis Machado for optimized functions on PowerPC. @@ -254,6 +270,9 @@ Alan Modra for his improvements for Linux on PowerPC. David Mosberger-Tang for contributing the port to Linux/Alpha (@code{alpha-@var{anything}-linux}). +@item +Wainer dos Santos Moschetta for various fixes to powerpc. + @item Stephen Moshier for implementing some 128-bit long double format math functions. @@ -274,6 +293,9 @@ coverage of conformtest, merging the ports/ subdirectory into the @glibcadj{} main repository and his direction as FSF Project Steward for @theglibc{}. +@item +Marko Myllynen for various fixes. + @item Szabolcs Nagy for various fixes. @@ -363,6 +385,9 @@ Richard Stallman for his contribution to the @code{getopt} function. @item Alfred M. Szmidt for various fixes. +@item +Chung-Lin Tang for contributing the Nios II port. + @item Ian Lance Taylor for contributing the port to the MIPS DECStation running Ultrix 4 (@code{mips-dec-ultrix4}). @@ -370,6 +395,9 @@ running Ultrix 4 (@code{mips-dec-ultrix4}). @item Samuel Thibault for improving the Hurd port. +@item +Pino Toscano for various fixes. + @item Tim Waugh for the implementation of the POSIX.2 @code{wordexp} function family. diff --git a/manual/install.texi b/manual/install.texi index b8deb9ceba..35948b1bbb 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -478,7 +478,7 @@ GCC 4.9 or newer GCC 4.9 or higher is required. In general it is recommended to use the newest version of the compiler that is known to work for building @theglibc{}, as newer compilers usually produce better code. As of -release time, GCC 6.3 is the newest compiler verified to work to build +release time, GCC 7.1 is the newest compiler verified to work to build @theglibc{}. For PowerPC 64-bits little-endian (powerpc64le), GCC 6.2 or higher is @@ -503,7 +503,7 @@ GNU @code{binutils} 2.25 or later You must use GNU @code{binutils} (as and ld) to build @theglibc{}. No other assembler or linker has the necessary functionality at the -moment. As of release time, GNU @code{binutils} 2.25 is the newest +moment. As of release time, GNU @code{binutils} 2.27 is the newest verified to work to build @theglibc{}. @item -- cgit v1.2.3 From 644d38570a860f3ed7d478c4ed8965a91e4621a1 Mon Sep 17 00:00:00 2001 From: Joseph Myers Date: Thu, 5 Oct 2017 15:58:13 +0000 Subject: Remove add-ons mechanism. glibc has an add-ons mechanism to allow additional software to be integrated into the glibc build. Such add-ons may be within the glibc source tree, or outside it at a path passed to the --enable-add-ons configure option. localedata and crypt were once add-ons, distributed in separate release tarballs, but long since stopped using that mechanism. Linuxthreads was always an add-on. Ports spent some time as an add-on with separate release tarballs, then was first moved into the glibc source tree, then had its sysdeps files moved into the main sysdeps hierarchy so the add-ons mechanism was no longer used. NPTL spent some time as an add-on in the main glibc tree before stopping using the add-on mechanism. libidn used to have separate release tarballs but no longer does so, but still uses the add-ons mechanism within the glibc source tree. Various other software has supported building with the add-ons mechanism at times in the past, but I don't think any is still widely used. Add-ons involve significant, little-used complexity in the glibc build system, and make it hard to understand what the space of possible glibc configurations is. This patch removes the add-ons mechanism. libidn is now built via the Subdirs mechanism to cause any configuration using sysdeps/unix/inet to build libidn; HAVE_LIBIDN (which effectively means shared libraries are available) is now defined via sysdeps/unix/inet/configure. Various references to add-ons around the source tree are removed (in the case of maint.texi, the example list of sysdeps directories is still very out of date). Externally maintained ports should now put their files in the normal sysdeps directory structure rather than being arranged as add-ons; they probably need to change e.g. elf.h anyway, rather than actually being able to work just as a drop-in subtree. Hurd libpthread should be arranged similarly to NPTL, so some files might go in a hurd-pthreads (or similar) top-level directory in glibc, while sysdeps files should go in the normal sysdeps directory structure (possibly in hurd or hurd-pthreads subdirectories, just as there are nptl subdirectories in the sysdeps tree). Tested for x86_64, and with build-many-glibcs.py. * configure.ac (--enable-add-ons): Remove option. (machine): Do not mention add-ons in comment. (LIBC_PRECONFIGURE): Likewise. (add_ons): Remove variable and sanity checks and logic to locate add-ons. (add_ons_automatic): Remove variable. (configured_add_ons): Likewise. (add_ons_sfx): Likewise. (add_ons_pfx): Likewise. (add_on_subdirs): Likewise. (sysnames_add_ons): Likewise. Remove loop over add-ons and consideration of add-ons in Implies handling. (sysdeps_add_ons): Likewise. * configure: Regenerated. * libidn/configure.ac: Remove. * libidn/configure: Likewise. * sysdeps/unix/inet/configure.ac: New file. * sysdeps/unix/inet/configure: New generated file. * sysdeps/unix/inet/Subdirs: Add libidn. * Makeconfig (sysdeps-srcdirs): Remove variable. (+sysdep_dirs): Do not include $(sysdeps-srcdirs). ($(common-objpfx)config.status): Do not depend on add-on files. ($(common-objpfx)shlib-versions.v.i): Do not mention add-ons in comment. (all-subdirs): Do not include $(add-on-subdirs). * Makefile (dist-prepare): Do not use $(sysdeps-add-ons). * config.make.in (add-ons): Remove variable. (add-on-subdirs): Likewise. (sysdeps-add-ons): Likewise. * manual/Makefile (add-chapters): Remove. ($(objpfx)texis): Do not depend on $(add-chapters). (nonexamples): Do not handle $(add-chapters). (examples): Do not handle $(add-ons). (chapters.% top-menu.%): Do not pass '$(add-chapters)' to libc-texinfo.sh. * manual/install.texi (Installation): Do not mention add-ons. (--enable-add-ons): Do not document configure option. * INSTALL: Regenerated. * manual/libc-texinfo.sh: Do not handle $2 add-ons argument. * manual/maint.texi (Hierarchy Conventions): Do not mention add-ons. * scripts/build-many-glibcs.py (Glibc.build_glibc): Do not use --enable-add-ons. * scripts/gen-sorted.awk: Do not handle Subdirs files from add-ons. * scripts/test-installation.pl: Do not handle glibc-compat add-on. * sysdeps/nptl/Makeconfig: Do not mention add-ons in comment. --- ChangeLog | 50 +++++++ INSTALL | 17 --- Makeconfig | 27 +--- Makefile | 3 +- NEWS | 4 + config.make.in | 3 - configure | 287 ++++------------------------------------- configure.ac | 268 ++++---------------------------------- libidn/configure | 16 --- libidn/configure.ac | 11 -- manual/Makefile | 9 +- manual/install.texi | 16 --- manual/libc-texinfo.sh | 17 +-- manual/maint.texi | 11 +- scripts/build-many-glibcs.py | 1 - scripts/gen-sorted.awk | 34 +---- scripts/test-installation.pl | 3 +- sysdeps/nptl/Makeconfig | 2 +- sysdeps/unix/inet/Subdirs | 1 + sysdeps/unix/inet/configure | 9 ++ sysdeps/unix/inet/configure.ac | 7 + 21 files changed, 142 insertions(+), 654 deletions(-) delete mode 100644 libidn/configure delete mode 100644 libidn/configure.ac create mode 100644 sysdeps/unix/inet/configure create mode 100644 sysdeps/unix/inet/configure.ac (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index b81a3ecd79..fc9637124c 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,53 @@ +2017-10-05 Joseph Myers + + * configure.ac (--enable-add-ons): Remove option. + (machine): Do not mention add-ons in comment. + (LIBC_PRECONFIGURE): Likewise. + (add_ons): Remove variable and sanity checks and logic to locate + add-ons. + (add_ons_automatic): Remove variable. + (configured_add_ons): Likewise. + (add_ons_sfx): Likewise. + (add_ons_pfx): Likewise. + (add_on_subdirs): Likewise. + (sysnames_add_ons): Likewise. Remove loop over add-ons and + consideration of add-ons in Implies handling. + (sysdeps_add_ons): Likewise. + * configure: Regenerated. + * libidn/configure.ac: Remove. + * libidn/configure: Likewise. + * sysdeps/unix/inet/configure.ac: New file. + * sysdeps/unix/inet/configure: New generated file. + * sysdeps/unix/inet/Subdirs: Add libidn. + * Makeconfig (sysdeps-srcdirs): Remove variable. + (+sysdep_dirs): Do not include $(sysdeps-srcdirs). + ($(common-objpfx)config.status): Do not depend on add-on files. + ($(common-objpfx)shlib-versions.v.i): Do not mention add-ons in + comment. + (all-subdirs): Do not include $(add-on-subdirs). + * Makefile (dist-prepare): Do not use $(sysdeps-add-ons). + * config.make.in (add-ons): Remove variable. + (add-on-subdirs): Likewise. + (sysdeps-add-ons): Likewise. + * manual/Makefile (add-chapters): Remove. + ($(objpfx)texis): Do not depend on $(add-chapters). + (nonexamples): Do not handle $(add-chapters). + (examples): Do not handle $(add-ons). + (chapters.% top-menu.%): Do not pass '$(add-chapters)' to + libc-texinfo.sh. + * manual/install.texi (Installation): Do not mention add-ons. + (--enable-add-ons): Do not document configure option. + * INSTALL: Regenerated. + * manual/libc-texinfo.sh: Do not handle $2 add-ons argument. + * manual/maint.texi (Hierarchy Conventions): Do not mention + add-ons. + * scripts/build-many-glibcs.py (Glibc.build_glibc): Do not use + --enable-add-ons. + * scripts/gen-sorted.awk: Do not handle Subdirs files from + add-ons. + * scripts/test-installation.pl: Do not handle glibc-compat add-on. + * sysdeps/nptl/Makeconfig: Do not mention add-ons in comment. + 2017-10-05 Andreas Schwab [BZ #15142] diff --git a/INSTALL b/INSTALL index 4d9024c667..bc972b2ffa 100644 --- a/INSTALL +++ b/INSTALL @@ -5,11 +5,6 @@ Before you do anything else, you should read the FAQ at . It answers common questions and describes problems you may experience with compilation and installation. - Features can be added to the GNU C Library via "add-on" bundles. -These are separate tar files, which you unpack into the top level of the -source tree. Then you give 'configure' the '--enable-add-ons' option to -activate them, and they will be compiled into the library. - You will need recent versions of several GNU tools: definitely GCC and GNU Make, and possibly others. *Note Tools for Compilation::, below. @@ -69,18 +64,6 @@ will be used, and CFLAGS sets optimization options for the compiler. this option if you want to compile the GNU C Library with a newer set of kernel headers than the ones found in '/usr/include'. -'--enable-add-ons[=LIST]' - Specify add-on packages to include in the build. If this option is - specified with no list, it enables all the add-on packages it finds - in the main source directory; this is the default behavior. You - may specify an explicit list of add-ons to use in LIST, separated - by spaces or commas (if you use spaces, remember to quote them from - the shell). Each add-on in LIST can be an absolute directory name - or can be a directory name relative to the main source directory, - or relative to the build directory (that is, the current working - directory). For example, - '--enable-add-ons=nptl,../glibc-libidn-VERSION'. - '--enable-kernel=VERSION' This option is currently only useful on GNU/Linux systems. The VERSION parameter should have the form X.Y.Z and describes the diff --git a/Makeconfig b/Makeconfig index 0ceff92a3d..5ff6f2e29a 100644 --- a/Makeconfig +++ b/Makeconfig @@ -59,15 +59,7 @@ stack-align-test-flags = # `configure' writes a definition of `config-sysdirs' in `config.make'. sysdirs := $(foreach D,$(config-sysdirs),$(firstword $(filter /%,$D) $(..)$D)) -# Add-ons that contribute sysdeps trees get added to the include list -# after sysdeps/generic. This makes #include work right -# to find specific add-on files without assuming the add-on directory name. -# It also means that headers can go into an add-on's base directory -# instead of the add-on needing a sysdeps/generic of its own. -sysdeps-srcdirs := $(foreach add-on,$(sysdeps-add-ons),\ - $(firstword $(filter /%,$(add-on)) \ - $(..)$(add-on))) -+sysdep_dirs = $(sysdirs) $(sysdeps-srcdirs) ++sysdep_dirs = $(sysdirs) ifdef objdir +sysdep_dirs := $(objdir) $(+sysdep_dirs) endif @@ -81,8 +73,8 @@ $(common-objpfx)config.make: $(common-objpfx)config.status \ $(..)config.make.in $(..)config.h.in cd $(&2; \ echo Run \`configure\' to configure it before building. >&2; \ @@ -1077,8 +1062,6 @@ endif ifeq ($(sysd-sorted-done),t) -include $(common-objpfx)soversions.mk ifndef avoid-generated -# This lets add-ons give more-specific matches that override defaults -# in the top-level file. $(common-objpfx)shlib-versions.v.i: \ $(wildcard $(+sysdep_dirs:=/shlib-versions) \ $(subdir-srcdirs:=/shlib-versions)) \ @@ -1186,7 +1169,7 @@ all-subdirs = csu assert ctype locale intl catgets math setjmp signal \ grp pwd posix io termios resource misc socket sysvipc gmon \ gnulib iconv iconvdata wctype manual shadow gshadow po argp \ crypt localedata timezone rt conform debug mathvec support \ - $(add-on-subdirs) dlfcn elf + dlfcn elf ifndef avoid-generated # sysd-sorted itself will contain rules making the sysd-sorted target diff --git a/Makefile b/Makefile index 3e0ae6f43b..869fe70475 100644 --- a/Makefile +++ b/Makefile @@ -351,8 +351,7 @@ files-for-dist := README INSTALL configure ChangeLog NEWS # Regenerate stuff, then error if these things are not committed yet. dist-prepare: $(files-for-dist) - conf=`find sysdeps $(addsuffix /sysdeps,$(sysdeps-add-ons)) \ - -name configure`; \ + conf=`find sysdeps -name configure`; \ $(MAKE) $$conf && \ git diff --stat HEAD -- $^ $$conf \ | $(AWK) '{ print; rc=1 } END { exit rc }' diff --git a/NEWS b/NEWS index a034cd8476..464d1ce9b3 100644 --- a/NEWS +++ b/NEWS @@ -50,6 +50,10 @@ Deprecated and removed features, and other changes affecting compatibility: platforms where it was previously the same, this changes the C++ name mangling for interfaces involving this type. +* The add-ons mechanism for building additional packages at the same time as + glibc has been removed. The --enable-add-ons configure option is now + ignored. + Changes to build and runtime requirements: [Add changes to build and runtime requirements here] diff --git a/config.make.in b/config.make.in index ea7a42cc19..fd2dbc91d1 100644 --- a/config.make.in +++ b/config.make.in @@ -90,9 +90,6 @@ build-pic-default= @libc_cv_pic_default@ build-pie-default= @libc_cv_pie_default@ build-profile = @profile@ build-static-nss = @static_nss@ -add-ons = @add_ons@ -add-on-subdirs = @add_on_subdirs@ -sysdeps-add-ons = @sysdeps_add_ons@ cross-compiling = @cross_compiling@ force-install = @force_install@ link-obsolete-rpc = @link_obsolete_rpc@ diff --git a/configure b/configure index a7c72eaaa6..c2c9c72d5a 100755 --- a/configure +++ b/configure @@ -659,7 +659,6 @@ LN_S INSTALL_DATA INSTALL_SCRIPT INSTALL_PROGRAM -sysdeps_add_ons sysnames submachine multi_arch @@ -667,8 +666,6 @@ no_stack_protector stack_protector libc_cv_ssp base_machine -add_on_subdirs -add_ons have_tunables build_pt_chown build_nscd @@ -771,7 +768,6 @@ enable_timezone_tools enable_hardcoded_path_in_tests enable_stackguard_randomization enable_lock_elision -enable_add_ons enable_hidden_plt enable_bind_now enable_stack_protector @@ -806,8 +802,7 @@ CPP CXX CXXFLAGS CCC' -ac_subdirs_all=' -' +ac_subdirs_all='' # Initialize some variables set by options. ac_init_help= @@ -1434,9 +1429,6 @@ Optional Features: number at program start --enable-lock-elision=yes/no Enable lock elision for pthread mutexes by default - --enable-add-ons[=DIRS...] - configure and build add-ons in DIR1,DIR2,... search - for add-ons if no parameter given --disable-hidden-plt do not hide internal function calls to avoid PLT --enable-bind-now disable lazy relocations in DSOs --enable-stack-protector=[yes|no|all|strong] @@ -3415,14 +3407,6 @@ if test "$enable_lock_elision" = yes ; then fi -# Check whether --enable-add-ons was given. -if test "${enable_add_ons+set}" = set; then : - enableval=$enable_add_ons; -else - enable_add_ons=yes -fi - - # Check whether --enable-hidden-plt was given. if test "${enable_hidden_plt+set}" = set; then : enableval=$enable_hidden_plt; hidden=$enableval @@ -3785,7 +3769,7 @@ config_os="`echo $config_os | sed 's/^unknown-//'`" # Some configurations imply other options. elf=yes -# The configure fragment of an add-on port can modify these to supplement +# The configure fragment of a port can modify these to supplement # or override the table in the case statement below. No fragment should # ever change the config_* variables, however. machine=$config_machine @@ -3823,143 +3807,6 @@ $as_echo_n "checking for sysdeps preconfigure fragments... " >&6; } $as_echo "" >&6; } fi -subdirs="$subdirs " - - -case "$enable_add_ons" in -''|no) add_ons= ;; -yes|'*') - add_ons=`cd $srcdir && ls -d 2> /dev/null */configure */sysdeps | - sed 's@/[^/]*$@@' | sort | uniq` - add_ons_automatic=yes - ;; -*) add_ons=`echo "$enable_add_ons" | sed 's/,/ /g'` - add_ons_automatic=no ;; -esac - -configured_add_ons= -add_ons_sfx= -add_ons_pfx= -if test x"$add_ons" != x; then - for f in $add_ons; do - # Some sanity checks - case "$f" in - crypt) - as_fn_error $? " -*** It seems that you're using an old \`crypt' add-on. crypt is now -*** part of glibc and using the old add-on will not work with this -*** release. Start again with fresh sources and without the old -*** \`crypt' add-on." "$LINENO" 5 - ;; - localedata) - as_fn_error $? " -*** It seems that you're using an old \`localedata' add-on. localedata -*** is now part of glibc and using the old add-on will not work with -*** this release. Start again with fresh sources and without the old -*** \`localedata' add-on." "$LINENO" 5 - ;; - esac - done - - # Now source each add-on's configure fragment. - # The fragments can use $srcdir/$libc_add_on to find themselves, - # and test $add_ons_automatic to see if they were explicitly requested. - # A fragment can clear (or even change) $libc_add_on to affect - # whether it goes into the list to be actually used in the build. - use_add_ons= - for libc_add_on in $add_ons; do - # Test whether such a directory really exists. - # It can be absolute, or relative to $srcdir, or relative to the build dir. - case "$libc_add_on" in - /*) - libc_add_on_srcdir=$libc_add_on - ;; - *) - test -d "$srcdir/$libc_add_on" || { - if test -d "$libc_add_on"; then - libc_add_on="`pwd`/$libc_add_on" - else - as_fn_error $? "add-on directory \"$libc_add_on\" does not exist" "$LINENO" 5 - fi - } - libc_add_on_srcdir=$srcdir/$libc_add_on - ;; - esac - - libc_add_on_frag=$libc_add_on_srcdir/configure - libc_add_on_canonical= - libc_add_on_config_subdirs= - if test -r "$libc_add_on_frag"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: running configure fragment for add-on $libc_add_on" >&5 -$as_echo "$as_me: running configure fragment for add-on $libc_add_on" >&6;} - libc_add_on_canonical=unknown - libc_add_on_subdirs= - . "$libc_add_on_frag" - test -z "$libc_add_on" || { - configured_add_ons="$configured_add_ons $libc_add_on" - if test "x$libc_add_on_canonical" = xunknown; then - as_fn_error $? "fragment must set \$libc_add_on_canonical" "$LINENO" 5 - fi - for d in $libc_add_on_subdirs; do - case "$libc_add_on" in - /*) subdir_srcdir="$libc_add_on" ;; - *) subdir_srcdir="\$(..)$libc_add_on" ;; - esac - case "$d" in - .) - d="${libc_add_on_canonical:-$libc_add_on}" - ;; - /*) - subdir_srcdir="$d" - ;; - *) - subdir_srcdir="$subdir_srcdir/$d" - ;; - esac - d=`echo "$d" | sed 's@/*$@@;s@^.*/@@'` - add_on_subdirs="$add_on_subdirs $d" - test "$subdir_srcdir" = "\$(..)$d" || config_vars="$config_vars -$d-srcdir = $subdir_srcdir" - done - for d in $libc_add_on_config_subdirs; do - case "$d" in - /*) as_fn_error $? "fragment uses absolute path in \$libc_add_on_config_subdirs" "$LINENO" 5 ;; - esac - if test ! -d "$libc_add_on_srcdir/$d"; then - as_fn_error $? "fragment wants to configure missing directory $d" "$LINENO" 5 - fi - case "$libc_add_on" in - /*) as_fn_error $? "relative path required for add-on using \$libc_add_on_config_subdirs" "$LINENO" 5 ;; - esac - subdirs="$subdirs $libc_add_on/$d" - done - } - fi - if test -n "$libc_add_on"; then - if frags=`ls -d $libc_add_on_srcdir/sysdeps/*/preconfigure 2> /dev/null` -then - { $as_echo "$as_me:${as_lineno-$LINENO}: checking add-on $libc_add_on for preconfigure fragments" >&5 -$as_echo_n "checking add-on $libc_add_on for preconfigure fragments... " >&6; } - for frag in $frags; do - name=`echo "$frag" | sed 's@/[^/]*$@@;s@^.*/@@'` - echo $ECHO_N "$name $ECHO_C" >&6 - . "$frag" - done - { $as_echo "$as_me:${as_lineno-$LINENO}: result: " >&5 -$as_echo "" >&6; } -fi - use_add_ons="$use_add_ons $libc_add_on" - add_ons_pfx="$add_ons_pfx $libc_add_on/" - test -z "$libc_add_on_canonical" || - add_ons_sfx="$add_ons_sfx /$libc_add_on_canonical" - fi - done - # Use echo to strip excess whitespace. - add_ons="`echo $use_add_ons`" -fi - - - ### ### By using the undocumented --enable-hacker-mode option for configure @@ -4231,7 +4078,6 @@ done # Find what sysdep directories exist. -sysnames_add_ons= sysnames= for b in $base ''; do for m0 in $mach ''; do @@ -4240,34 +4086,21 @@ for b in $base ''; do for o in /$ostry ''; do test "$o" = / && continue for m in $mach ''; do - for d in $add_ons_pfx ''; do - for a in $add_ons_sfx ''; do - try_suffix="$m0$b$v$o$m" - if test -n "$try_suffix"; then - try_srcdir="${srcdir}/" - case "$d" in - /*) try_srcdir= ;; - esac - try="${d}sysdeps$try_suffix$a" - test -n "$enable_debug_configure" && - echo "$0 DEBUG: try $try" >&2 - if test -d "$try_srcdir$try"; then - sysnames="$sysnames $try" - { test -n "$o" || test -n "$b"; } && os_used=t - { test -n "$m" || test -n "$m0"; } && machine_used=t - case x${m0:-$m} in - x*/$submachine) submachine_used=t ;; - esac - if test -n "$d"; then - case "$sysnames_add_ons" in - *" $d "*) ;; - *|'') sysnames_add_ons="$sysnames_add_ons $d" ;; - esac - fi - fi - fi - done - done + try_suffix="$m0$b$v$o$m" + if test -n "$try_suffix"; then + try_srcdir="${srcdir}/" + try="sysdeps$try_suffix" + test -n "$enable_debug_configure" && + echo "$0 DEBUG: try $try" >&2 + if test -d "$try_srcdir$try"; then + sysnames="$sysnames $try" + { test -n "$o" || test -n "$b"; } && os_used=t + { test -n "$m" || test -n "$m0"; } && machine_used=t + case x${m0:-$m} in + x*/$submachine) submachine_used=t ;; + esac + fi + fi done done done @@ -4276,7 +4109,7 @@ done # If the assembler supports gnu_indirect_function symbol type and the # architecture supports multi-arch, we enable multi-arch by default. -case $sysnames_add_ons$sysnames in +case $sysnames in *"$multi_arch_d"*) ;; *) @@ -4339,24 +4172,15 @@ while test $# -gt 0; do eval "${implies_type}=\"\$${implies_type} \$name_base/\$x\"" found=yes fi - for d in $add_ons_pfx ''; do - try="${d}sysdeps/$x" - case $d in - /*) try_srcdir= ;; - *) try_srcdir=$srcdir/ ;; - esac - test -n "$enable_debug_configure" && - echo "DEBUG: $name $implies_file $x try($d) {$try_srcdir}$try" >&2 - if test $try != $xsrcdir$name_base/$x && test -d $try_srcdir$try; - then - eval "${implies_type}=\"\$${implies_type} \$try\"" - found=yes - case "$sysnames_add_ons" in - *" $d "*) ;; - *|'') sysnames_add_ons="$sysnames_add_ons $d" ;; - esac - fi - done + try="sysdeps/$x" + try_srcdir=$srcdir/ + test -n "$enable_debug_configure" && + echo "DEBUG: $name $implies_file $x try() {$try_srcdir}$try" >&2 + if test $try != $xsrcdir$name_base/$x && test -d $try_srcdir$try; + then + eval "${implies_type}=\"\$${implies_type} \$try\"" + found=yes + fi if test $found = no; then { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $name/$implies_file specifies nonexistent $x" >&5 $as_echo "$as_me: WARNING: $name/$implies_file specifies nonexistent $x" >&2;} @@ -4394,63 +4218,6 @@ sysnames="$names $default_sysnames" { $as_echo "$as_me:${as_lineno-$LINENO}: result: $default_sysnames" >&5 $as_echo "$default_sysnames" >&6; } -# Collect the list of add-ons that supply partial sysdeps trees. -sysdeps_add_ons= -for add_on in $add_ons; do - case "$add_on" in - /*) xsrcdir= ;; - *) xsrcdir="$srcdir/" ;; - esac - - test -d "$xsrcdir$add_on/sysdeps" || { - case "$configured_add_ons " in - *" $add_on "*) ;; - *|'') - as_fn_error $? "add-on $add_on has no configure fragment or sysdeps tree" "$LINENO" 5 - ;; - esac - continue - } - - sysdeps_add_ons="$sysdeps_add_ons $add_on" - case "$sysnames_add_ons" in - *" $add_on/ "*) ;; - *|'') - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: add-on $add_on contributed no sysdeps directories" >&5 -$as_echo "$as_me: WARNING: add-on $add_on contributed no sysdeps directories" >&2;} - continue ;; - esac - - found=no - for d in $sysnames; do - case "$d" in - $add_on/sysdeps/*) ;; - *) continue ;; - esac - (cd "$xsrcdir$d" && for f in *[!~]; do - case "$f" in - sys|bits) - for ff in $f/*.h; do - test -d "$ff" || { test -e "$ff" && exit 88; } - done - ;; - *) - test -d "$f" || { test -e "$f" && exit 88; } - ;; - esac - done) - if test $? -eq 88; then - found=yes - break - fi - done - if test $found = no; then - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: add-on $add_on contributed no useful sysdeps directories" >&5 -$as_echo "$as_me: WARNING: add-on $add_on contributed no useful sysdeps directories" >&2;} - fi -done - - ### Locate tools. diff --git a/configure.ac b/configure.ac index 4b83ae5a09..195e81acfd 100644 --- a/configure.ac +++ b/configure.ac @@ -209,13 +209,6 @@ if test "$enable_lock_elision" = yes ; then AC_DEFINE(ENABLE_LOCK_ELISION) fi -dnl Generic infrastructure for drop-in additions to libc. -AC_ARG_ENABLE([add-ons], - AC_HELP_STRING([--enable-add-ons@<:@=DIRS...@:>@], - [configure and build add-ons in DIR1,DIR2,... - search for add-ons if no parameter given]), - , [enable_add_ons=yes]) - AC_ARG_ENABLE([hidden-plt], AC_HELP_STRING([--disable-hidden-plt], [do not hide internal function calls to avoid PLT]), @@ -483,7 +476,7 @@ config_os="`echo $config_os | sed 's/^unknown-//'`" # Some configurations imply other options. elf=yes -# The configure fragment of an add-on port can modify these to supplement +# The configure fragment of a port can modify these to supplement # or override the table in the case statement below. No fragment should # ever change the config_* variables, however. machine=$config_machine @@ -506,138 +499,9 @@ AC_ARG_WITH([cpu], # check below. libc_config_ok=no -dnl Let sysdeps/*/preconfigure act here, like they can in add-ons. +dnl Let sysdeps/*/preconfigure act here. LIBC_PRECONFIGURE([$srcdir], [for sysdeps]) -dnl Having this here, though empty, makes sure that if add-ons' fragments -dnl do AC_CONFIG_SUBDIRS([some-dir]), which just sets $subdirs, then -dnl our AC_OUTPUT will actually use it. -AC_CONFIG_SUBDIRS() - -case "$enable_add_ons" in -''|no) add_ons= ;; -yes|'*') - add_ons=`cd $srcdir && ls -d 2> /dev/null */configure */sysdeps | - sed 's@/[[^/]]*$@@' | sort | uniq` - add_ons_automatic=yes - ;; -*) add_ons=`echo "$enable_add_ons" | sed 's/,/ /g'` - add_ons_automatic=no ;; -esac - -configured_add_ons= -add_ons_sfx= -add_ons_pfx= -if test x"$add_ons" != x; then - for f in $add_ons; do - # Some sanity checks - case "$f" in - crypt) - AC_MSG_ERROR([ -*** It seems that you're using an old \`crypt' add-on. crypt is now -*** part of glibc and using the old add-on will not work with this -*** release. Start again with fresh sources and without the old -*** \`crypt' add-on.]) - ;; - localedata) - AC_MSG_ERROR([ -*** It seems that you're using an old \`localedata' add-on. localedata -*** is now part of glibc and using the old add-on will not work with -*** this release. Start again with fresh sources and without the old -*** \`localedata' add-on.]) - ;; - esac - done - - # Now source each add-on's configure fragment. - # The fragments can use $srcdir/$libc_add_on to find themselves, - # and test $add_ons_automatic to see if they were explicitly requested. - # A fragment can clear (or even change) $libc_add_on to affect - # whether it goes into the list to be actually used in the build. - use_add_ons= - for libc_add_on in $add_ons; do - # Test whether such a directory really exists. - # It can be absolute, or relative to $srcdir, or relative to the build dir. - case "$libc_add_on" in - /*) - libc_add_on_srcdir=$libc_add_on - ;; - *) - test -d "$srcdir/$libc_add_on" || { - if test -d "$libc_add_on"; then - libc_add_on="`pwd`/$libc_add_on" - else - AC_MSG_ERROR(add-on directory \"$libc_add_on\" does not exist) - fi - } - libc_add_on_srcdir=$srcdir/$libc_add_on - ;; - esac - - libc_add_on_frag=$libc_add_on_srcdir/configure - libc_add_on_canonical= - libc_add_on_config_subdirs= - if test -r "$libc_add_on_frag"; then - AC_MSG_NOTICE(running configure fragment for add-on $libc_add_on) - libc_add_on_canonical=unknown - libc_add_on_subdirs= - . "$libc_add_on_frag" - test -z "$libc_add_on" || { - configured_add_ons="$configured_add_ons $libc_add_on" - if test "x$libc_add_on_canonical" = xunknown; then - AC_MSG_ERROR(fragment must set \$libc_add_on_canonical) - fi - for d in $libc_add_on_subdirs; do - case "$libc_add_on" in - /*) subdir_srcdir="$libc_add_on" ;; - *) subdir_srcdir="\$(..)$libc_add_on" ;; - esac - case "$d" in - .) - d="${libc_add_on_canonical:-$libc_add_on}" - ;; - /*) - subdir_srcdir="$d" - ;; - *) - subdir_srcdir="$subdir_srcdir/$d" - ;; - esac - d=`echo "$d" | sed 's@/*$@@;s@^.*/@@'` - add_on_subdirs="$add_on_subdirs $d" - test "$subdir_srcdir" = "\$(..)$d" || config_vars="$config_vars -$d-srcdir = $subdir_srcdir" - done - for d in $libc_add_on_config_subdirs; do - case "$d" in - /*) AC_MSG_ERROR(dnl -fragment uses absolute path in \$libc_add_on_config_subdirs) ;; - esac - if test ! -d "$libc_add_on_srcdir/$d"; then - AC_MSG_ERROR(fragment wants to configure missing directory $d) - fi - case "$libc_add_on" in - /*) AC_MSG_ERROR(dnl -relative path required for add-on using \$libc_add_on_config_subdirs) ;; - esac - subdirs="$subdirs $libc_add_on/$d" - done - } - fi - if test -n "$libc_add_on"; then - LIBC_PRECONFIGURE([$libc_add_on_srcdir], [add-on $libc_add_on for]) - use_add_ons="$use_add_ons $libc_add_on" - add_ons_pfx="$add_ons_pfx $libc_add_on/" - test -z "$libc_add_on_canonical" || - add_ons_sfx="$add_ons_sfx /$libc_add_on_canonical" - fi - done - # Use echo to strip excess whitespace. - add_ons="`echo $use_add_ons`" -fi -AC_SUBST(add_ons) -AC_SUBST(add_on_subdirs) - ### ### By using the undocumented --enable-hacker-mode option for configure @@ -853,7 +717,6 @@ dnl We are done with glob and regexp uses of [ and ]; return to autoconf. changequote([,])dnl # Find what sysdep directories exist. -sysnames_add_ons= sysnames= for b in $base ''; do for m0 in $mach ''; do @@ -862,34 +725,21 @@ for b in $base ''; do for o in /$ostry ''; do test "$o" = / && continue for m in $mach ''; do - for d in $add_ons_pfx ''; do - for a in $add_ons_sfx ''; do - try_suffix="$m0$b$v$o$m" - if test -n "$try_suffix"; then - try_srcdir="${srcdir}/" - case "$d" in - /*) try_srcdir= ;; - esac - try="${d}sysdeps$try_suffix$a" - test -n "$enable_debug_configure" && - echo "$0 [DEBUG]: try $try" >&2 - if test -d "$try_srcdir$try"; then - sysnames="$sysnames $try" - { test -n "$o" || test -n "$b"; } && os_used=t - { test -n "$m" || test -n "$m0"; } && machine_used=t - case x${m0:-$m} in - x*/$submachine) submachine_used=t ;; - esac - if test -n "$d"; then - case "$sysnames_add_ons" in - *" $d "*) ;; - *|'') sysnames_add_ons="$sysnames_add_ons $d" ;; - esac - fi - fi - fi - done - done + try_suffix="$m0$b$v$o$m" + if test -n "$try_suffix"; then + try_srcdir="${srcdir}/" + try="sysdeps$try_suffix" + test -n "$enable_debug_configure" && + echo "$0 [DEBUG]: try $try" >&2 + if test -d "$try_srcdir$try"; then + sysnames="$sysnames $try" + { test -n "$o" || test -n "$b"; } && os_used=t + { test -n "$m" || test -n "$m0"; } && machine_used=t + case x${m0:-$m} in + x*/$submachine) submachine_used=t ;; + esac + fi + fi done done done @@ -898,7 +748,7 @@ done # If the assembler supports gnu_indirect_function symbol type and the # architecture supports multi-arch, we enable multi-arch by default. -case $sysnames_add_ons$sysnames in +case $sysnames in *"$multi_arch_d"*) ;; *) @@ -960,24 +810,15 @@ while test $# -gt 0; do eval "${implies_type}=\"\$${implies_type} \$name_base/\$x\"" found=yes fi - for d in $add_ons_pfx ''; do - try="${d}sysdeps/$x" - case $d in - /*) try_srcdir= ;; - *) try_srcdir=$srcdir/ ;; - esac - test -n "$enable_debug_configure" && - echo "[DEBUG]: $name $implies_file $x try($d) {$try_srcdir}$try" >&2 - if test $try != $xsrcdir$name_base/$x && test -d $try_srcdir$try; - then - eval "${implies_type}=\"\$${implies_type} \$try\"" - found=yes - case "$sysnames_add_ons" in - *" $d "*) ;; - *|'') sysnames_add_ons="$sysnames_add_ons $d" ;; - esac - fi - done + try="sysdeps/$x" + try_srcdir=$srcdir/ + test -n "$enable_debug_configure" && + echo "[DEBUG]: $name $implies_file $x try() {$try_srcdir}$try" >&2 + if test $try != $xsrcdir$name_base/$x && test -d $try_srcdir$try; + then + eval "${implies_type}=\"\$${implies_type} \$try\"" + found=yes + fi if test $found = no; then AC_MSG_WARN($name/$implies_file specifies nonexistent $x) fi @@ -1015,61 +856,6 @@ AC_SUBST(sysnames) # The other names were emitted during the scan. AC_MSG_RESULT($default_sysnames) -# Collect the list of add-ons that supply partial sysdeps trees. -sysdeps_add_ons= -for add_on in $add_ons; do - case "$add_on" in - /*) xsrcdir= ;; - *) xsrcdir="$srcdir/" ;; - esac - - test -d "$xsrcdir$add_on/sysdeps" || { - case "$configured_add_ons " in - *" $add_on "*) ;; - *|'') - AC_MSG_ERROR(add-on $add_on has no configure fragment or sysdeps tree) - ;; - esac - continue - } - - sysdeps_add_ons="$sysdeps_add_ons $add_on" - case "$sysnames_add_ons" in - *" $add_on/ "*) ;; - *|'') - AC_MSG_WARN(add-on $add_on contributed no sysdeps directories) - continue ;; - esac - - found=no - for d in $sysnames; do - case "$d" in - $add_on/sysdeps/*) ;; - *) continue ;; - esac - (cd "$xsrcdir$d" && for f in *[[!~]]; do - case "$f" in - sys|bits) - for ff in $f/*.h; do - test -d "$ff" || { test -e "$ff" && exit 88; } - done - ;; - *) - test -d "$f" || { test -e "$f" && exit 88; } - ;; - esac - done) - if test $? -eq 88; then - found=yes - break - fi - done - if test $found = no; then - AC_MSG_WARN(add-on $add_on contributed no useful sysdeps directories) - fi -done -AC_SUBST(sysdeps_add_ons) - ### Locate tools. diff --git a/libidn/configure b/libidn/configure deleted file mode 100644 index 8195663d96..0000000000 --- a/libidn/configure +++ /dev/null @@ -1,16 +0,0 @@ -# This file is generated from configure.ac by Autoconf. DO NOT EDIT! - -libc_add_on_canonical= - -if test "$shared" = yes; then : - - libc_add_on_subdirs=. - - # Get this defined in config.h for main source code to test. - $as_echo "#define HAVE_LIBIDN 1" >>confdefs.h - - -else - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: libidn add-on ignored with --disable-shared" >&5 -$as_echo "$as_me: WARNING: libidn add-on ignored with --disable-shared" >&2;} -fi diff --git a/libidn/configure.ac b/libidn/configure.ac deleted file mode 100644 index 16740ffb92..0000000000 --- a/libidn/configure.ac +++ /dev/null @@ -1,11 +0,0 @@ -dnl glibc configure fragment for libidn add-on -GLIBC_PROVIDES dnl See aclocal.m4 in the top level source directory. - -libc_add_on_canonical= - -AS_IF([test "$shared" = yes], [ - libc_add_on_subdirs=. - - # Get this defined in config.h for main source code to test. - AC_DEFINE([HAVE_LIBIDN]) -], [AC_MSG_WARN([libidn add-on ignored with --disable-shared])]) diff --git a/manual/Makefile b/manual/Makefile index 4ed63a8ef3..3b4c7c934a 100644 --- a/manual/Makefile +++ b/manual/Makefile @@ -40,20 +40,18 @@ chapters = $(addsuffix .texi, \ resource setjmp signal startup process ipc job \ nss users sysinfo conf crypt debug threads \ probes tunables) -add-chapters = $(wildcard $(foreach d, $(add-ons), ../$d/$d.texi)) appendices = lang.texi header.texi install.texi maint.texi platform.texi \ contrib.texi licenses = freemanuals.texi lgpl-2.1.texi fdl-1.3.texi -include $(objpfx)texis -$(objpfx)texis: texis.awk $(chapters) $(add-chapters) $(appendices) $(licenses) +$(objpfx)texis: texis.awk $(chapters) $(appendices) $(licenses) $(make-target-directory) $(AWK) -f $^ > $@.T mv -f $@.T $@ -nonexamples = $(filter-out $(add-chapters) %.c.texi, $(texis)) -examples = $(filter-out $(foreach d, $(add-ons), ../$d/%.c.texi), \ - $(filter %.c.texi, $(texis))) +nonexamples = $(filter-out %.c.texi, $(texis)) +examples = $(filter %.c.texi, $(texis)) # Generated files directly included from libc.texinfo. libc-texi-generated = chapters.texi top-menu.texi dir-add.texi \ @@ -69,7 +67,6 @@ texis-path := $(filter-out $(libc-texi-generated) summary.texi $(examples), \ chapters.% top-menu.%: libc-texinfo.sh $(texis-path) Makefile AWK=$(AWK) $(SHELL) $< $(objpfx) \ '$(chapters)' \ - '$(add-chapters)' \ '$(appendices) $(licenses)' diff --git a/manual/install.texi b/manual/install.texi index 35948b1bbb..96b988e829 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -13,11 +13,6 @@ Before you do anything else, you should read the FAQ at questions and describes problems you may experience with compilation and installation. -Features can be added to @theglibc{} via @dfn{add-on} bundles. These are -separate tar files, which you unpack into the top level of the source -tree. Then you give @code{configure} the @samp{--enable-add-ons} option -to activate them, and they will be compiled into the library. - You will need recent versions of several GNU tools: definitely GCC and GNU Make, and possibly others. @xref{Tools for Compilation}, below. @@ -95,17 +90,6 @@ occasionally happen in this case. You can also use this option if you want to compile @theglibc{} with a newer set of kernel headers than the ones found in @file{/usr/include}. -@item --enable-add-ons[=@var{list}] -Specify add-on packages to include in the build. If this option is -specified with no list, it enables all the add-on packages it finds in -the main source directory; this is the default behavior. You may -specify an explicit list of add-ons to use in @var{list}, separated by -spaces or commas (if you use spaces, remember to quote them from the -shell). Each add-on in @var{list} can be an absolute directory name -or can be a directory name relative to the main source directory, or -relative to the build directory (that is, the current working directory). -For example, @samp{--enable-add-ons=nptl,../glibc-libidn-@var{version}}. - @item --enable-kernel=@var{version} This option is currently only useful on @gnulinuxsystems{}. The @var{version} parameter should have the form X.Y.Z and describes the diff --git a/manual/libc-texinfo.sh b/manual/libc-texinfo.sh index 4d0a52213b..aea27f9e94 100644 --- a/manual/libc-texinfo.sh +++ b/manual/libc-texinfo.sh @@ -31,26 +31,11 @@ collect_nodes () { $AWK -f tsort.awk | sed 's/_/ /g' } -# Emit "@set ADD-ON" for each add-on contributing a manual chapter. -for addon in $2; do - addon=`basename $addon .texi` - echo >&3 "@set $addon" -done - collect_nodes $1 | build_menu -if [ -n "$2" ]; then - - { echo; echo 'Add-ons'; echo; } >&4 - - egrep '^(@c )?@node.*Top' `echo $2 /dev/null | tr ' ' '\n' | sort` | - cut -d, -f1 | sed 's/@c //;s/@node //' | build_menu - -fi - { echo; echo 'Appendices'; echo; } >&4 -collect_nodes $3 | build_menu +collect_nodes $2 | build_menu exec 3>&- 4>&- 5>&- diff --git a/manual/maint.texi b/manual/maint.texi index 473ab162f0..fce06bfa88 100644 --- a/manual/maint.texi +++ b/manual/maint.texi @@ -381,19 +381,10 @@ identical @file{irix6.2} and @file{irix6.3} directories, by removing trailing suffixes starting with a period. As an example, here is the complete list of directories that would be -tried for the configuration @w{@samp{i686-linux-gnu}} (with the -@file{crypt} and @file{linuxthreads} add-on): +tried for the configuration @w{@samp{i686-linux-gnu}}: @smallexample sysdeps/i386/elf -crypt/sysdeps/unix -linuxthreads/sysdeps/unix/sysv/linux -linuxthreads/sysdeps/pthread -linuxthreads/sysdeps/unix/sysv -linuxthreads/sysdeps/unix -linuxthreads/sysdeps/i386/i686 -linuxthreads/sysdeps/i386 -linuxthreads/sysdeps/pthread/no-cmpxchg sysdeps/unix/sysv/linux/i386 sysdeps/unix/sysv/linux sysdeps/gnu diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py index 8ce1c6f7f9..a6c01f9cc4 100755 --- a/scripts/build-many-glibcs.py +++ b/scripts/build-many-glibcs.py @@ -1342,7 +1342,6 @@ class Glibc(object): cmdlist.create_copy_dir(srcdir, srcdir_copy) cfg_cmd = [os.path.join(srcdir_copy, 'configure'), '--prefix=/usr', - '--enable-add-ons', '--enable-profile', '--build=%s' % self.ctx.build_triplet, '--host=%s' % self.triplet, diff --git a/scripts/gen-sorted.awk b/scripts/gen-sorted.awk index 922eafcc2d..ecbed9ef43 100755 --- a/scripts/gen-sorted.awk +++ b/scripts/gen-sorted.awk @@ -46,36 +46,10 @@ type == "Subdirs" && NF == 2 && $1 == "inhibit" { type == "Subdirs" && thisdir { all[cnt++] = thisdir; - if (FILENAME ~ (srcpfx ? /^\.\.\/sysdeps\// : /^sysdeps\//) \ - || system("test -d " srcpfx thisdir) == 0) { - # This Subdirs file is in the main source tree, - # or this subdirectory exists in the main source tree. - this_srcdir = srcpfx thisdir - } - else { - # The Subdirs file comes from an add-on that should have the subdirectory. - dir = FILENAME; - do - sub(/\/[^/]+$/, "", dir); - while (dir !~ /\/sysdeps$/); - sub(/\/sysdeps$/, "", dir); - if (system("test -d " dir "/" thisdir) == 0) - dir = dir "/" thisdir; - else { - sub(/\/[^/]+$/, "", dir); - if (system("test -d " dir "/" thisdir) == 0) - dir = dir "/" thisdir; - else { - print FILENAME ":" FNR ":", "cannot find", thisdir > "/dev/stderr"; - exit 2 - } - } - file = dir "/Depend"; - if (srcpfx) - sub(/^\.\.\//, "", dir); - if (dir !~ /^\/.*$/) - dir = "$(..)" dir; - print thisdir "-srcdir", ":=", dir; + this_srcdir = srcpfx thisdir + if (system("test -d " this_srcdir) != 0) { + print FILENAME ":" FNR ":", "cannot find", this_srcdir > "/dev/stderr"; + exit 2 } file = this_srcdir "/Depend"; if (system("test -f " file) == 0) { diff --git a/scripts/test-installation.pl b/scripts/test-installation.pl index c5b9fdefd2..45c666b0a2 100755 --- a/scripts/test-installation.pl +++ b/scripts/test-installation.pl @@ -116,14 +116,13 @@ while () { # Filter out some libraries we don't want to link: # - nss_ldap since it's not yet available # - libdb1 since it conflicts with libdb - # - libnss1_* from glibc-compat add-on # - libthread_db since it contains unresolved references # - it's just a test NSS module # - We don't provide the libgcc so we don't test it # - libmvec if it wasn't built next if ($build_mathvec == 0 && $name eq "mvec"); if ($name ne "nss_ldap" && $name ne "db1" - && !($name =~/^nss1_/) && $name ne "thread_db" + && $name ne "thread_db" && $name ne "nss_test1" && $name ne "libgcc_s") { $link_libs .= " -l$name"; $versions{$name} = $version; diff --git a/sysdeps/nptl/Makeconfig b/sysdeps/nptl/Makeconfig index d31b77680f..3516d51df3 100644 --- a/sysdeps/nptl/Makeconfig +++ b/sysdeps/nptl/Makeconfig @@ -16,7 +16,7 @@ # License along with the GNU C Library; if not, see # . -# Makeconfig fragment for NPTL add-on. +# Makeconfig fragment for NPTL. # This gets included at the end of the main glibc Makeconfig. have-thread-library = yes diff --git a/sysdeps/unix/inet/Subdirs b/sysdeps/unix/inet/Subdirs index 0a02dd4447..1223e43a37 100644 --- a/sysdeps/unix/inet/Subdirs +++ b/sysdeps/unix/inet/Subdirs @@ -6,3 +6,4 @@ nis nscd nss streams +libidn diff --git a/sysdeps/unix/inet/configure b/sysdeps/unix/inet/configure new file mode 100644 index 0000000000..fd0afad16d --- /dev/null +++ b/sysdeps/unix/inet/configure @@ -0,0 +1,9 @@ +# This file is generated from configure.ac by Autoconf. DO NOT EDIT! + +if test "$shared" = yes; then : + + # Get this defined in config.h for main source code to test. + $as_echo "#define HAVE_LIBIDN 1" >>confdefs.h + + +fi diff --git a/sysdeps/unix/inet/configure.ac b/sysdeps/unix/inet/configure.ac new file mode 100644 index 0000000000..e9b3443733 --- /dev/null +++ b/sysdeps/unix/inet/configure.ac @@ -0,0 +1,7 @@ +dnl glibc configure fragment for sysdeps/unix/inet. +GLIBC_PROVIDES dnl See aclocal.m4 in the top level source directory. + +AS_IF([test "$shared" = yes], [ + # Get this defined in config.h for main source code to test. + AC_DEFINE([HAVE_LIBIDN]) +]) -- cgit v1.2.3 From a306c790a835894c22d076a04a9924d3daeb9462 Mon Sep 17 00:00:00 2001 From: Siddhesh Poyarekar Date: Thu, 16 Nov 2017 11:49:26 +0530 Subject: Prefer https for Sourceware links Update all sourceware links to https. The website redirects everything to https anyway so let the web server do a bit less work. The only reference that remains unchanged is the one in the old ChangeLog, since it didn't seem worth changing it. * NEWS: Update sourceware link to https. * configure.ac: Likewise. * crypt/md5test-giant.c: Likewise. * dlfcn/bug-atexit1.c: Likewise. * dlfcn/bug-atexit2.c: Likewise. * localedata/README: Likewise. * malloc/tst-mallocfork.c: Likewise. * manual/install.texi: Likewise. * nptl/tst-pthread-getattr.c: Likewise. * stdio-common/tst-fgets.c: Likewise. * stdio-common/tst-fwrite.c: Likewise. * sunrpc/Makefile: Likewise. * sysdeps/arm/armv7/multiarch/memcpy_impl.S: Likewise. * wcsmbs/tst-mbrtowc2.c: Likewise. * configure: Regenerate. * INSTALL: Regenerate. --- ChangeLog | 19 +++++++++++++++++++ INSTALL | 4 ++-- NEWS | 2 +- configure | 14 +++++++------- configure.ac | 6 +++--- crypt/md5test-giant.c | 2 +- dlfcn/bug-atexit1.c | 2 +- dlfcn/bug-atexit2.c | 2 +- localedata/README | 2 +- malloc/tst-mallocfork.c | 2 +- manual/install.texi | 4 ++-- nptl/tst-pthread-getattr.c | 4 ++-- stdio-common/tst-fgets.c | 2 +- stdio-common/tst-fwrite.c | 2 +- sunrpc/Makefile | 2 +- sysdeps/arm/armv7/multiarch/memcpy_impl.S | 2 +- wcsmbs/tst-mbrtowc2.c | 3 ++- 17 files changed, 47 insertions(+), 27 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index d7d81aceb7..c5a5c9c502 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,22 @@ +2017-11-16 Siddhesh Poyarekar + + * INSTALL: Update sourceware link to https. + * NEWS: Likewise. + * configure: Likewise. + * configure.ac: Likewise. + * crypt/md5test-giant.c: Likewise. + * dlfcn/bug-atexit1.c: Likewise. + * dlfcn/bug-atexit2.c: Likewise. + * localedata/README: Likewise. + * malloc/tst-mallocfork.c: Likewise. + * manual/install.texi: Likewise. + * nptl/tst-pthread-getattr.c: Likewise. + * stdio-common/tst-fgets.c: Likewise. + * stdio-common/tst-fwrite.c: Likewise. + * sunrpc/Makefile: Likewise. + * sysdeps/arm/armv7/multiarch/memcpy_impl.S: Likewise. + * wcsmbs/tst-mbrtowc2.c: Likewise. + 2017-11-15 Martin Sebor * misc/sys/cdefs.h (__attribute_nonstring__): New macro. diff --git a/INSTALL b/INSTALL index bc972b2ffa..35e82fbca9 100644 --- a/INSTALL +++ b/INSTALL @@ -2,7 +2,7 @@ Installing the GNU C Library **************************** Before you do anything else, you should read the FAQ at -. It answers common questions and +. It answers common questions and describes problems you may experience with compilation and installation. You will need recent versions of several GNU tools: definitely GCC @@ -541,7 +541,7 @@ remain unfixed for all eternity, if not longer. It is a good idea to verify that the problem has not already been reported. Bugs are documented in two places: The file 'BUGS' describes a number of well known bugs and the central GNU C Library bug tracking -system has a WWW interface at . The +system has a WWW interface at . The WWW interface gives you access to open and closed reports. A closed report normally includes a patch or a hint on solving the problem. diff --git a/NEWS b/NEWS index 520db40982..1c5da21327 100644 --- a/NEWS +++ b/NEWS @@ -2,7 +2,7 @@ GNU C Library NEWS -- history of user-visible changes. Copyright (C) 1992-2017 Free Software Foundation, Inc. See the end for copying conditions. -Please send GNU C library bug reports via +Please send GNU C library bug reports via using `glibc' in the "product" field. Version 2.27 diff --git a/configure b/configure index c8697d9b1a..d9d9243238 100755 --- a/configure +++ b/configure @@ -2,7 +2,7 @@ # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.69 for GNU C Library (see version.h). # -# Report bugs to . +# Report bugs to . # # # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc. @@ -266,7 +266,7 @@ fi $as_echo "$0: be upgraded to zsh 4.3.4 or later." else $as_echo "$0: Please tell bug-autoconf@gnu.org and -$0: http://sourceware.org/bugzilla/ about your system, +$0: https://sourceware.org/bugzilla/ about your system, $0: including any error possibly output before this $0: message. Then install a modern shell, or manually run $0: the script under such a shell if you do have one." @@ -581,7 +581,7 @@ PACKAGE_NAME='GNU C Library' PACKAGE_TARNAME='glibc' PACKAGE_VERSION='(see version.h)' PACKAGE_STRING='GNU C Library (see version.h)' -PACKAGE_BUGREPORT='http://sourceware.org/bugzilla/' +PACKAGE_BUGREPORT='https://sourceware.org/bugzilla/' PACKAGE_URL='http://www.gnu.org/software/glibc/' ac_unique_file="include/features.h" @@ -1494,7 +1494,7 @@ Some influential environment variables: Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. -Report bugs to . +Report bugs to . GNU C Library home page: . General help using GNU software: . _ACEOF @@ -2188,12 +2188,12 @@ _ACEOF # We require GCC, and by default use its preprocessor. Override AC_PROG_CPP # here to work around the Autoconf issue discussed in -# . +# . # AC_PROG_CPP # We require GCC. Override _AC_PROG_CC_C89 here to work around the Autoconf # issue discussed in -# . +# . @@ -7286,7 +7286,7 @@ $config_headers Configuration commands: $config_commands -Report bugs to . +Report bugs to . GNU C Library home page: . General help using GNU software: ." diff --git a/configure.ac b/configure.ac index 9f25c9fa0f..8ebc490a55 100644 --- a/configure.ac +++ b/configure.ac @@ -1,6 +1,6 @@ dnl Process this file with autoconf to produce a configure script. dnl Note we do not use AC_PREREQ here! See aclocal.m4 for what we use instead. -AC_INIT([GNU C Library], [(see version.h)], [http://sourceware.org/bugzilla/], [glibc]) +AC_INIT([GNU C Library], [(see version.h)], [https://sourceware.org/bugzilla/], [glibc]) AC_CONFIG_SRCDIR([include/features.h]) AC_CONFIG_HEADERS([config.h]) AC_CONFIG_AUX_DIR([scripts]) @@ -19,7 +19,7 @@ AC_DEFUN([_AC_INCLUDES_DEFAULT_REQUIREMENTS], # We require GCC, and by default use its preprocessor. Override AC_PROG_CPP # here to work around the Autoconf issue discussed in -# . +# . AC_DEFUN([AC_PROG_CPP], [AC_REQUIRE([AC_PROG_CC])dnl AC_ARG_VAR([CPP], [C preprocessor])dnl @@ -36,7 +36,7 @@ AC_SUBST(CPP)dnl # We require GCC. Override _AC_PROG_CC_C89 here to work around the Autoconf # issue discussed in -# . +# . AC_DEFUN([_AC_PROG_CC_C89], [[$1]]) dnl This is here so we can set $subdirs directly based on configure fragments. diff --git a/crypt/md5test-giant.c b/crypt/md5test-giant.c index 4655a74db8..a7634eeb35 100644 --- a/crypt/md5test-giant.c +++ b/crypt/md5test-giant.c @@ -1,4 +1,4 @@ -/* Testcase for http://sourceware.org/bugzilla/show_bug.cgi?id=14090. +/* Testcase for https://sourceware.org/bugzilla/show_bug.cgi?id=14090. Copyright (C) 2012-2017 Free Software Foundation, Inc. This file is part of the GNU C Library. diff --git a/dlfcn/bug-atexit1.c b/dlfcn/bug-atexit1.c index e2d1d2f776..fdfb6bc0b7 100644 --- a/dlfcn/bug-atexit1.c +++ b/dlfcn/bug-atexit1.c @@ -1,5 +1,5 @@ /* Derived from a test case in - http://sourceware.org/bugzilla/show_bug.cgi?id=1158. */ + https://sourceware.org/bugzilla/show_bug.cgi?id=1158. */ #include #include #include diff --git a/dlfcn/bug-atexit2.c b/dlfcn/bug-atexit2.c index 15e9f7aa01..279804dcf2 100644 --- a/dlfcn/bug-atexit2.c +++ b/dlfcn/bug-atexit2.c @@ -1,5 +1,5 @@ /* Derived from a test case in - http://sourceware.org/bugzilla/show_bug.cgi?id=1158. */ + https://sourceware.org/bugzilla/show_bug.cgi?id=1158. */ #include #include #include diff --git a/localedata/README b/localedata/README index 3e828986c1..e844719180 100644 --- a/localedata/README +++ b/localedata/README @@ -42,7 +42,7 @@ especially in the section describing the `setlocale' function. All problems should be reported using - http://sourceware.org/bugzilla/ + https://sourceware.org/bugzilla/ One more note: the `POSIX' locale definition is not meant to be used diff --git a/malloc/tst-mallocfork.c b/malloc/tst-mallocfork.c index f90ce94887..4ff6ec09f4 100644 --- a/malloc/tst-mallocfork.c +++ b/malloc/tst-mallocfork.c @@ -1,5 +1,5 @@ /* Derived from the test case in - http://sourceware.org/bugzilla/show_bug.cgi?id=838. */ + https://sourceware.org/bugzilla/show_bug.cgi?id=838. */ #include #include #include diff --git a/manual/install.texi b/manual/install.texi index 96b988e829..f1fa28c937 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -9,7 +9,7 @@ @appendix Installing @theglibc{} Before you do anything else, you should read the FAQ at -@url{http://sourceware.org/glibc/wiki/FAQ}. It answers common +@url{https://sourceware.org/glibc/wiki/FAQ}. It answers common questions and describes problems you may experience with compilation and installation. @@ -613,7 +613,7 @@ reported. Bugs are documented in two places: The file @file{BUGS} describes a number of well known bugs and the central @glibcadj{} bug tracking system has a WWW interface at -@url{http://sourceware.org/bugzilla/}. The WWW +@url{https://sourceware.org/bugzilla/}. The WWW interface gives you access to open and closed reports. A closed report normally includes a patch or a hint on solving the problem. diff --git a/nptl/tst-pthread-getattr.c b/nptl/tst-pthread-getattr.c index 86719f97ab..27c60c84de 100644 --- a/nptl/tst-pthread-getattr.c +++ b/nptl/tst-pthread-getattr.c @@ -35,8 +35,8 @@ results in a test case failure. To avoid these problems, we cap the stack size to one less than 8M. See the following mailing list threads for more information about this problem: - - . */ + + . */ #define MAX_STACK_SIZE (8192 * 1024 - 1) static size_t pagesize; diff --git a/stdio-common/tst-fgets.c b/stdio-common/tst-fgets.c index 0aa9030e3a..912b7068eb 100644 --- a/stdio-common/tst-fgets.c +++ b/stdio-common/tst-fgets.c @@ -1,5 +1,5 @@ /* Derived from the test case in - http://sourceware.org/bugzilla/show_bug.cgi?id=713. */ + https://sourceware.org/bugzilla/show_bug.cgi?id=713. */ #include static int diff --git a/stdio-common/tst-fwrite.c b/stdio-common/tst-fwrite.c index 2986c8932a..1db10d0f6a 100644 --- a/stdio-common/tst-fwrite.c +++ b/stdio-common/tst-fwrite.c @@ -1,5 +1,5 @@ /* Derived from the test case in - http://sourceware.org/bugzilla/show_bug.cgi?id=1078. */ + https://sourceware.org/bugzilla/show_bug.cgi?id=1078. */ #include #include #include diff --git a/sunrpc/Makefile b/sunrpc/Makefile index 125d538208..f1b8323e93 100644 --- a/sunrpc/Makefile +++ b/sunrpc/Makefile @@ -173,7 +173,7 @@ cross-rpcgen-objs := $(addprefix $(objpfx)cross-,$(rpcgen-objs)) # When generic makefile support for build system programs is # available, it should replace this code. See -# . +# . $(cross-rpcgen-objs): $(objpfx)cross-%.o: %.c $(before-compile) $(BUILD_CC) $($(basename $( #include #include -- cgit v1.2.3 From 7b0fb8706c07a38c8bb9999bd4c4563f07e2495f Mon Sep 17 00:00:00 2001 From: Siddhesh Poyarekar Date: Thu, 16 Nov 2017 12:08:52 +0530 Subject: Fix botched up regeneration in the last commit --- ChangeLog | 8 +++++--- INSTALL | 5 +++-- 2 files changed, 8 insertions(+), 5 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index c5a5c9c502..f6bdc7a50d 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,8 +1,8 @@ 2017-11-16 Siddhesh Poyarekar - * INSTALL: Update sourceware link to https. - * NEWS: Likewise. - * configure: Likewise. + * INSTALL: Fix botched up regeneration. + + * NEWS: Update sourceare link to https. * configure.ac: Likewise. * crypt/md5test-giant.c: Likewise. * dlfcn/bug-atexit1.c: Likewise. @@ -16,6 +16,8 @@ * sunrpc/Makefile: Likewise. * sysdeps/arm/armv7/multiarch/memcpy_impl.S: Likewise. * wcsmbs/tst-mbrtowc2.c: Likewise. + * configure: Regenerate. + * INSTALL: Regenerate. 2017-11-15 Martin Sebor diff --git a/INSTALL b/INSTALL index 35e82fbca9..ac8db74a0f 100644 --- a/INSTALL +++ b/INSTALL @@ -2,8 +2,9 @@ Installing the GNU C Library **************************** Before you do anything else, you should read the FAQ at -. It answers common questions and -describes problems you may experience with compilation and installation. +. It answers common questions +and describes problems you may experience with compilation and +installation. You will need recent versions of several GNU tools: definitely GCC and GNU Make, and possibly others. *Note Tools for Compilation::, -- cgit v1.2.3 From 1faaf7035cabda101e1d6653bff7a539f201db91 Mon Sep 17 00:00:00 2001 From: Juro Bystricky Date: Thu, 30 Nov 2017 21:21:15 +0000 Subject: plural.c: improve reproducibility There is a subtle non-determinism when building glibc. This depends on whether the glibc is built using the distibuted file intl/plural.c or built using the generated file intl/plural.c. These two files (intl/plural.c generated vs. distributed) are slightly different, hence we may end up with slightly different libraries. Originally, having "bison" installed was optional. So if "bison" was not present, we always built libraries with the distributed plural.c. If bison was installed, we *** may have *** replaced the distributed file plural.c with a new plural.c generated from plural.y. if the timestamps triggered this rule: plural.c plural.y $(BISON) $(BISONFLAGS) $@ $^ Given that timestamps are not preserved in GIT repositories, the above rule is not reliable without explicitly touching plural.c or plural.y. In other words, the rule may or may not have fired. In summary: there are two distinct sources of non-determinism: 1. Having "bison" installed or not 2. Having "bison" installed but timestamps poorly defined. This patch fixes this by requiring "bison" being installed and by always generating intl/plural.c from intl/plural.y. (This is achieved by simply removing checked-in intl/plural.c) [BZ #22432] * configure.ac (BISON): Require to be present. * configure: Regenerated. * intl/Makefile (generated): Add plural.c. [$(BISON) != no]: Make code unconditional. (plural.c): Change rule to $(objpfx)plural.c. ($(objpfx)plural.o): Depend on $(objpfx)plural.c. * intl/plural.c: Remove. * manual/install.texi (Tools for Compilation): Document bison as required. * INSTALL: Regenerated. --- ChangeLog | 14 + INSTALL | 10 +- NEWS | 3 +- configure | 128 ++-- configure.ac | 7 +- intl/Makefile | 9 +- intl/plural.c | 2011 --------------------------------------------------- manual/install.texi | 14 +- 8 files changed, 98 insertions(+), 2098 deletions(-) delete mode 100644 intl/plural.c (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 8c042b8be6..4380b73a03 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,17 @@ +2017-11-30 Juro Bystricky + + [BZ #22432] + * configure.ac (BISON): Require to be present. + * configure: Regenerated. + * intl/Makefile (generated): Add plural.c. + [$(BISON) != no]: Make code unconditional. + (plural.c): Change rule to $(objpfx)plural.c. + ($(objpfx)plural.o): Depend on $(objpfx)plural.c. + * intl/plural.c: Remove. + * manual/install.texi (Tools for Compilation): Document bison as + required. + * INSTALL: Regenerated. + 2017-11-30 Joseph Myers * sysdeps/m68k/m680x0/fpu/s_llrint.c: Include diff --git a/INSTALL b/INSTALL index ac8db74a0f..d1a34c2a90 100644 --- a/INSTALL +++ b/INSTALL @@ -462,6 +462,11 @@ build the GNU C Library: version 4.1.3 is the newest verified to work to build the GNU C Library. + * GNU 'bison' 2.7 or later + + 'bison' is used to generate the 'yacc' parser code in the 'intl' + subdirectory. + * Perl 5 Perl is not required, but it is used if present to test the @@ -481,11 +486,6 @@ and if you change any of the message translation files you will need * GNU 'gettext' 0.10.36 or later -If you wish to regenerate the 'yacc' parser code in the 'intl' -subdirectory you will need - - * GNU 'bison' 2.7 or later - You may also need these packages if you upgrade your source tree using patches, although we try to avoid this. diff --git a/NEWS b/NEWS index f3fdf9aec5..48af4acaea 100644 --- a/NEWS +++ b/NEWS @@ -79,7 +79,8 @@ Deprecated and removed features, and other changes affecting compatibility: Changes to build and runtime requirements: - [Add changes to build and runtime requirements here] +* bison version 2.7 or later is required to generate code in the 'intl' + subdirectory. Security related changes: diff --git a/configure b/configure index d9d9243238..caba102846 100755 --- a/configure +++ b/configure @@ -632,7 +632,6 @@ ASFLAGS_config libc_cv_cc_with_libunwind libc_cv_insert libc_cv_protected_data -BISON INSTALL_INFO PERL BASH_SHELL @@ -645,6 +644,7 @@ PYTHON PYTHON_PROG AUTOCONF NM +BISON AWK SED MAKEINFO @@ -4940,6 +4940,69 @@ if test $ac_verc_fail = yes; then critic_missing="$critic_missing gawk" fi +for ac_prog in bison +do + # Extract the first word of "$ac_prog", so it can be a program name with args. +set dummy $ac_prog; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_prog_BISON+:} false; then : + $as_echo_n "(cached) " >&6 +else + if test -n "$BISON"; then + ac_cv_prog_BISON="$BISON" # Let the user override the test. +else +as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_prog_BISON="$ac_prog" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + +fi +fi +BISON=$ac_cv_prog_BISON +if test -n "$BISON"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $BISON" >&5 +$as_echo "$BISON" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + + test -n "$BISON" && break +done + +if test -z "$BISON"; then + ac_verc_fail=yes +else + # Found it, now check the version. + { $as_echo "$as_me:${as_lineno-$LINENO}: checking version of $BISON" >&5 +$as_echo_n "checking version of $BISON... " >&6; } + ac_prog_version=`$BISON --version 2>&1 | sed -n 's/^.*bison (GNU Bison) \([0-9]*\.[0-9.]*\).*$/\1/p'` + case $ac_prog_version in + '') ac_prog_version="v. ?.??, bad"; ac_verc_fail=yes;; + 2.7*|[3-9].*|[1-9][0-9]*) + ac_prog_version="$ac_prog_version, ok"; ac_verc_fail=no;; + *) ac_prog_version="$ac_prog_version, bad"; ac_verc_fail=yes;; + + esac + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_prog_version" >&5 +$as_echo "$ac_prog_version" >&6; } +fi +if test $ac_verc_fail = yes; then + critic_missing="$critic_missing bison" +fi + { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $CC is sufficient to build libc" >&5 $as_echo_n "checking if $CC is sufficient to build libc... " >&6; } @@ -5399,69 +5462,6 @@ $as_echo "no" >&6; } fi -for ac_prog in bison -do - # Extract the first word of "$ac_prog", so it can be a program name with args. -set dummy $ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_BISON+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$BISON"; then - ac_cv_prog_BISON="$BISON" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_BISON="$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -BISON=$ac_cv_prog_BISON -if test -n "$BISON"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $BISON" >&5 -$as_echo "$BISON" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$BISON" && break -done - -if test -z "$BISON"; then - ac_verc_fail=yes -else - # Found it, now check the version. - { $as_echo "$as_me:${as_lineno-$LINENO}: checking version of $BISON" >&5 -$as_echo_n "checking version of $BISON... " >&6; } - ac_prog_version=`$BISON --version 2>&1 | sed -n 's/^.*bison (GNU Bison) \([0-9]*\.[0-9.]*\).*$/\1/p'` - case $ac_prog_version in - '') ac_prog_version="v. ?.??, bad"; ac_verc_fail=yes;; - 2.7*|[3-9].*|[1-9][0-9]*) - ac_prog_version="$ac_prog_version, ok"; ac_verc_fail=no;; - *) ac_prog_version="$ac_prog_version, bad"; ac_verc_fail=yes;; - - esac - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_prog_version" >&5 -$as_echo "$ac_prog_version" >&6; } -fi -if test $ac_verc_fail = yes; then - BISON=no -fi - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for .set assembler directive" >&5 $as_echo_n "checking for .set assembler directive... " >&6; } diff --git a/configure.ac b/configure.ac index 8ebc490a55..9707ae4d8e 100644 --- a/configure.ac +++ b/configure.ac @@ -939,6 +939,9 @@ AC_CHECK_PROG_VER(SED, sed, --version, AC_CHECK_PROG_VER(AWK, gawk, --version, [GNU Awk[^0-9]*\([0-9][0-9.]*\)], [3.1.[2-9]*|3.[2-9]*|[4-9]*], critic_missing="$critic_missing gawk") +AC_CHECK_PROG_VER(BISON, bison, --version, + [bison (GNU Bison) \([0-9]*\.[0-9.]*\)], + [2.7*|[3-9].*|[1-9][0-9]*], critic_missing="$critic_missing bison") AC_CACHE_CHECK([if $CC is sufficient to build libc], libc_cv_compiler_ok, [ AC_TRY_COMPILE([], [ @@ -1070,10 +1073,6 @@ if test "$PERL" != no && fi AC_PATH_PROG(INSTALL_INFO, install-info, no, $PATH:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin) -AC_CHECK_PROG_VER(BISON, bison, --version, - [bison (GNU Bison) \([0-9]*\.[0-9.]*\)], - [2.7*|[3-9].*|[1-9][0-9]*], - BISON=no) AC_CACHE_CHECK(for .set assembler directive, libc_cv_asm_set_directive, [dnl cat > conftest.s <. */ - -/* As a special exception, you may create a larger work that contains - part or all of the Bison parser skeleton and distribute that work - under terms of your choice, so long as that work isn't itself a - parser generator using the skeleton or a modified version thereof - as a parser skeleton. Alternatively, if you modify or redistribute - the parser skeleton itself, you may (at your option) remove this - special exception, which will cause the skeleton and the resulting - Bison output files to be licensed under the GNU General Public - License without this special exception. - - This special exception was added by the Free Software Foundation in - version 2.2 of Bison. */ - -/* C LALR(1) parser skeleton written by Richard Stallman, by - simplifying the original so-called "semantic" parser. */ - -/* All symbols defined below should begin with yy or YY, to avoid - infringing on user name space. This should be done even for local - variables, as they might otherwise be expanded by user macros. - There are some unavoidable exceptions within include files to - define necessary library symbols; they are noted "INFRINGES ON - USER NAME SPACE" below. */ - -/* Identify Bison output. */ -#define YYBISON 1 - -/* Bison version. */ -#define YYBISON_VERSION "2.7" - -/* Skeleton name. */ -#define YYSKELETON_NAME "yacc.c" - -/* Pure parsers. */ -#define YYPURE 2 - -/* Push parsers. */ -#define YYPUSH 0 - -/* Pull parsers. */ -#define YYPULL 1 - - -/* Substitute the variable and function names. */ -#define yyparse __gettextparse -#define yylex __gettextlex -#define yyerror __gettexterror -#define yylval __gettextlval -#define yychar __gettextchar -#define yydebug __gettextdebug -#define yynerrs __gettextnerrs - -/* Copy the first part of user declarations. */ -/* Line 371 of yacc.c */ -#line 1 "plural.y" - -/* Expression parsing for plural form selection. - Copyright (C) 2000-2017 Free Software Foundation, Inc. - Written by Ulrich Drepper , 2000. - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU Lesser General Public License as published by - the Free Software Foundation; either version 2.1 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public License - along with this program. If not, see . */ - -/* For bison < 2.0, the bison generated parser uses alloca. AIX 3 forces us - to put this declaration at the beginning of the file. The declaration in - bison's skeleton file comes too late. This must come before - because may include arbitrary system headers. - This can go away once the AM_INTL_SUBDIR macro requires bison >= 2.0. */ -#if defined _AIX && !defined __GNUC__ - #pragma alloca -#endif - -#ifdef HAVE_CONFIG_H -# include -#endif - -#include -#include -#include -#include "plural-exp.h" - -/* The main function generated by the parser is called __gettextparse, - but we want it to be called PLURAL_PARSE. */ -#ifndef _LIBC -# define __gettextparse PLURAL_PARSE -#endif - - -/* Line 371 of yacc.c */ -#line 119 "plural.c" - -# ifndef YY_NULL -# if defined __cplusplus && 201103L <= __cplusplus -# define YY_NULL nullptr -# else -# define YY_NULL 0 -# endif -# endif - -/* Enabling verbose error messages. */ -#ifdef YYERROR_VERBOSE -# undef YYERROR_VERBOSE -# define YYERROR_VERBOSE 1 -#else -# define YYERROR_VERBOSE 0 -#endif - - -/* Enabling traces. */ -#ifndef YYDEBUG -# define YYDEBUG 0 -#endif -#if YYDEBUG -extern int __gettextdebug; -#endif - -/* Tokens. */ -#ifndef YYTOKENTYPE -# define YYTOKENTYPE - /* Put the tokens into the symbol table, so that GDB and other debuggers - know about them. */ - enum yytokentype { - EQUOP2 = 258, - CMPOP2 = 259, - ADDOP2 = 260, - MULOP2 = 261, - NUMBER = 262 - }; -#endif -/* Tokens. */ -#define EQUOP2 258 -#define CMPOP2 259 -#define ADDOP2 260 -#define MULOP2 261 -#define NUMBER 262 - - - -#if ! defined YYSTYPE && ! defined YYSTYPE_IS_DECLARED -typedef union YYSTYPE -{ -/* Line 387 of yacc.c */ -#line 49 "plural.y" - - unsigned long int num; - enum expression_operator op; - struct expression *exp; - - -/* Line 387 of yacc.c */ -#line 180 "plural.c" -} YYSTYPE; -# define YYSTYPE_IS_TRIVIAL 1 -# define yystype YYSTYPE /* obsolescent; will be withdrawn */ -# define YYSTYPE_IS_DECLARED 1 -#endif - - -#ifdef YYPARSE_PARAM -#if defined __STDC__ || defined __cplusplus -int __gettextparse (void *YYPARSE_PARAM); -#else -int __gettextparse (); -#endif -#else /* ! YYPARSE_PARAM */ -#if defined __STDC__ || defined __cplusplus -int __gettextparse (struct parse_args *arg); -#else -int __gettextparse (); -#endif -#endif /* ! YYPARSE_PARAM */ - - - -/* Copy the second part of user declarations. */ -/* Line 390 of yacc.c */ -#line 55 "plural.y" - -/* Prototypes for local functions. */ -static int yylex (YYSTYPE *lval, struct parse_args *arg); -static void yyerror (struct parse_args *arg, const char *str); - -/* Allocation of expressions. */ - -static struct expression * -new_exp (int nargs, enum expression_operator op, - struct expression * const *args) -{ - int i; - struct expression *newp; - - /* If any of the argument could not be malloc'ed, just return NULL. */ - for (i = nargs - 1; i >= 0; i--) - if (args[i] == NULL) - goto fail; - - /* Allocate a new expression. */ - newp = (struct expression *) malloc (sizeof (*newp)); - if (newp != NULL) - { - newp->nargs = nargs; - newp->operation = op; - for (i = nargs - 1; i >= 0; i--) - newp->val.args[i] = args[i]; - return newp; - } - - fail: - for (i = nargs - 1; i >= 0; i--) - FREE_EXPRESSION (args[i]); - - return NULL; -} - -static inline struct expression * -new_exp_0 (enum expression_operator op) -{ - return new_exp (0, op, NULL); -} - -static inline struct expression * -new_exp_1 (enum expression_operator op, struct expression *right) -{ - struct expression *args[1]; - - args[0] = right; - return new_exp (1, op, args); -} - -static struct expression * -new_exp_2 (enum expression_operator op, struct expression *left, - struct expression *right) -{ - struct expression *args[2]; - - args[0] = left; - args[1] = right; - return new_exp (2, op, args); -} - -static inline struct expression * -new_exp_3 (enum expression_operator op, struct expression *bexp, - struct expression *tbranch, struct expression *fbranch) -{ - struct expression *args[3]; - - args[0] = bexp; - args[1] = tbranch; - args[2] = fbranch; - return new_exp (3, op, args); -} - - -/* Line 390 of yacc.c */ -#line 284 "plural.c" - -#ifdef short -# undef short -#endif - -#ifdef YYTYPE_UINT8 -typedef YYTYPE_UINT8 yytype_uint8; -#else -typedef unsigned char yytype_uint8; -#endif - -#ifdef YYTYPE_INT8 -typedef YYTYPE_INT8 yytype_int8; -#elif (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -typedef signed char yytype_int8; -#else -typedef short int yytype_int8; -#endif - -#ifdef YYTYPE_UINT16 -typedef YYTYPE_UINT16 yytype_uint16; -#else -typedef unsigned short int yytype_uint16; -#endif - -#ifdef YYTYPE_INT16 -typedef YYTYPE_INT16 yytype_int16; -#else -typedef short int yytype_int16; -#endif - -#ifndef YYSIZE_T -# ifdef __SIZE_TYPE__ -# define YYSIZE_T __SIZE_TYPE__ -# elif defined size_t -# define YYSIZE_T size_t -# elif ! defined YYSIZE_T && (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -# include /* INFRINGES ON USER NAME SPACE */ -# define YYSIZE_T size_t -# else -# define YYSIZE_T unsigned int -# endif -#endif - -#define YYSIZE_MAXIMUM ((YYSIZE_T) -1) - -#ifndef YY_ -# if defined YYENABLE_NLS && YYENABLE_NLS -# if ENABLE_NLS -# include /* INFRINGES ON USER NAME SPACE */ -# define YY_(Msgid) dgettext ("bison-runtime", Msgid) -# endif -# endif -# ifndef YY_ -# define YY_(Msgid) Msgid -# endif -#endif - -/* Suppress unused-variable warnings by "using" E. */ -#if ! defined lint || defined __GNUC__ -# define YYUSE(E) ((void) (E)) -#else -# define YYUSE(E) /* empty */ -#endif - -/* Identity function, used to suppress warnings about constant conditions. */ -#ifndef lint -# define YYID(N) (N) -#else -#if (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -static int -YYID (int yyi) -#else -static int -YYID (yyi) - int yyi; -#endif -{ - return yyi; -} -#endif - -#if ! defined yyoverflow || YYERROR_VERBOSE - -/* The parser invokes alloca or malloc; define the necessary symbols. */ - -# ifdef YYSTACK_USE_ALLOCA -# if YYSTACK_USE_ALLOCA -# ifdef __GNUC__ -# define YYSTACK_ALLOC __builtin_alloca -# elif defined __BUILTIN_VA_ARG_INCR -# include /* INFRINGES ON USER NAME SPACE */ -# elif defined _AIX -# define YYSTACK_ALLOC __alloca -# elif defined _MSC_VER -# include /* INFRINGES ON USER NAME SPACE */ -# define alloca _alloca -# else -# define YYSTACK_ALLOC alloca -# if ! defined _ALLOCA_H && ! defined EXIT_SUCCESS && (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -# include /* INFRINGES ON USER NAME SPACE */ - /* Use EXIT_SUCCESS as a witness for stdlib.h. */ -# ifndef EXIT_SUCCESS -# define EXIT_SUCCESS 0 -# endif -# endif -# endif -# endif -# endif - -# ifdef YYSTACK_ALLOC - /* Pacify GCC's `empty if-body' warning. */ -# define YYSTACK_FREE(Ptr) do { /* empty */; } while (YYID (0)) -# ifndef YYSTACK_ALLOC_MAXIMUM - /* The OS might guarantee only one guard page at the bottom of the stack, - and a page size can be as small as 4096 bytes. So we cannot safely - invoke alloca (N) if N exceeds 4096. Use a slightly smaller number - to allow for a few compiler-allocated temporary stack slots. */ -# define YYSTACK_ALLOC_MAXIMUM 4032 /* reasonable circa 2006 */ -# endif -# else -# define YYSTACK_ALLOC YYMALLOC -# define YYSTACK_FREE YYFREE -# ifndef YYSTACK_ALLOC_MAXIMUM -# define YYSTACK_ALLOC_MAXIMUM YYSIZE_MAXIMUM -# endif -# if (defined __cplusplus && ! defined EXIT_SUCCESS \ - && ! ((defined YYMALLOC || defined malloc) \ - && (defined YYFREE || defined free))) -# include /* INFRINGES ON USER NAME SPACE */ -# ifndef EXIT_SUCCESS -# define EXIT_SUCCESS 0 -# endif -# endif -# ifndef YYMALLOC -# define YYMALLOC malloc -# if ! defined malloc && ! defined EXIT_SUCCESS && (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -void *malloc (YYSIZE_T); /* INFRINGES ON USER NAME SPACE */ -# endif -# endif -# ifndef YYFREE -# define YYFREE free -# if ! defined free && ! defined EXIT_SUCCESS && (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -void free (void *); /* INFRINGES ON USER NAME SPACE */ -# endif -# endif -# endif -#endif /* ! defined yyoverflow || YYERROR_VERBOSE */ - - -#if (! defined yyoverflow \ - && (! defined __cplusplus \ - || (defined YYSTYPE_IS_TRIVIAL && YYSTYPE_IS_TRIVIAL))) - -/* A type that is properly aligned for any stack member. */ -union yyalloc -{ - yytype_int16 yyss_alloc; - YYSTYPE yyvs_alloc; -}; - -/* The size of the maximum gap between one aligned stack and the next. */ -# define YYSTACK_GAP_MAXIMUM (sizeof (union yyalloc) - 1) - -/* The size of an array large to enough to hold all stacks, each with - N elements. */ -# define YYSTACK_BYTES(N) \ - ((N) * (sizeof (yytype_int16) + sizeof (YYSTYPE)) \ - + YYSTACK_GAP_MAXIMUM) - -# define YYCOPY_NEEDED 1 - -/* Relocate STACK from its old location to the new one. The - local variables YYSIZE and YYSTACKSIZE give the old and new number of - elements in the stack, and YYPTR gives the new location of the - stack. Advance YYPTR to a properly aligned location for the next - stack. */ -# define YYSTACK_RELOCATE(Stack_alloc, Stack) \ - do \ - { \ - YYSIZE_T yynewbytes; \ - YYCOPY (&yyptr->Stack_alloc, Stack, yysize); \ - Stack = &yyptr->Stack_alloc; \ - yynewbytes = yystacksize * sizeof (*Stack) + YYSTACK_GAP_MAXIMUM; \ - yyptr += yynewbytes / sizeof (*yyptr); \ - } \ - while (YYID (0)) - -#endif - -#if defined YYCOPY_NEEDED && YYCOPY_NEEDED -/* Copy COUNT objects from SRC to DST. The source and destination do - not overlap. */ -# ifndef YYCOPY -# if defined __GNUC__ && 1 < __GNUC__ -# define YYCOPY(Dst, Src, Count) \ - __builtin_memcpy (Dst, Src, (Count) * sizeof (*(Src))) -# else -# define YYCOPY(Dst, Src, Count) \ - do \ - { \ - YYSIZE_T yyi; \ - for (yyi = 0; yyi < (Count); yyi++) \ - (Dst)[yyi] = (Src)[yyi]; \ - } \ - while (YYID (0)) -# endif -# endif -#endif /* !YYCOPY_NEEDED */ - -/* YYFINAL -- State number of the termination state. */ -#define YYFINAL 9 -/* YYLAST -- Last index in YYTABLE. */ -#define YYLAST 54 - -/* YYNTOKENS -- Number of terminals. */ -#define YYNTOKENS 16 -/* YYNNTS -- Number of nonterminals. */ -#define YYNNTS 3 -/* YYNRULES -- Number of rules. */ -#define YYNRULES 13 -/* YYNRULES -- Number of states. */ -#define YYNSTATES 27 - -/* YYTRANSLATE(YYLEX) -- Bison symbol number corresponding to YYLEX. */ -#define YYUNDEFTOK 2 -#define YYMAXUTOK 262 - -#define YYTRANSLATE(YYX) \ - ((unsigned int) (YYX) <= YYMAXUTOK ? yytranslate[YYX] : YYUNDEFTOK) - -/* YYTRANSLATE[YYLEX] -- Bison symbol number corresponding to YYLEX. */ -static const yytype_uint8 yytranslate[] = -{ - 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 10, 2, 2, 2, 2, 5, 2, - 14, 15, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 12, 2, - 2, 2, 2, 3, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 13, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 4, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, - 2, 2, 2, 2, 2, 2, 1, 2, 6, 7, - 8, 9, 11 -}; - -#if YYDEBUG -/* YYPRHS[YYN] -- Index of the first RHS symbol of rule number YYN in - YYRHS. */ -static const yytype_uint8 yyprhs[] = -{ - 0, 0, 3, 5, 11, 15, 19, 23, 27, 31, - 35, 38, 40, 42 -}; - -/* YYRHS -- A `-1'-separated list of the rules' RHS. */ -static const yytype_int8 yyrhs[] = -{ - 17, 0, -1, 18, -1, 18, 3, 18, 12, 18, - -1, 18, 4, 18, -1, 18, 5, 18, -1, 18, - 6, 18, -1, 18, 7, 18, -1, 18, 8, 18, - -1, 18, 9, 18, -1, 10, 18, -1, 13, -1, - 11, -1, 14, 18, 15, -1 -}; - -/* YYRLINE[YYN] -- source line where rule number YYN was defined. */ -static const yytype_uint8 yyrline[] = -{ - 0, 152, 152, 160, 164, 168, 172, 176, 180, 184, - 188, 192, 196, 201 -}; -#endif - -#if YYDEBUG || YYERROR_VERBOSE || 0 -/* YYTNAME[SYMBOL-NUM] -- String name of the symbol SYMBOL-NUM. - First, the terminals, then, starting at YYNTOKENS, nonterminals. */ -static const char *const yytname[] = -{ - "$end", "error", "$undefined", "'?'", "'|'", "'&'", "EQUOP2", "CMPOP2", - "ADDOP2", "MULOP2", "'!'", "NUMBER", "':'", "'n'", "'('", "')'", - "$accept", "start", "exp", YY_NULL -}; -#endif - -# ifdef YYPRINT -/* YYTOKNUM[YYLEX-NUM] -- Internal token number corresponding to - token YYLEX-NUM. */ -static const yytype_uint16 yytoknum[] = -{ - 0, 256, 257, 63, 124, 38, 258, 259, 260, 261, - 33, 262, 58, 110, 40, 41 -}; -# endif - -/* YYR1[YYN] -- Symbol number of symbol that rule YYN derives. */ -static const yytype_uint8 yyr1[] = -{ - 0, 16, 17, 18, 18, 18, 18, 18, 18, 18, - 18, 18, 18, 18 -}; - -/* YYR2[YYN] -- Number of symbols composing right hand side of rule YYN. */ -static const yytype_uint8 yyr2[] = -{ - 0, 2, 1, 5, 3, 3, 3, 3, 3, 3, - 2, 1, 1, 3 -}; - -/* YYDEFACT[STATE-NAME] -- Default reduction number in state STATE-NUM. - Performed when YYTABLE doesn't specify something else to do. Zero - means the default is an error. */ -static const yytype_uint8 yydefact[] = -{ - 0, 0, 12, 11, 0, 0, 2, 10, 0, 1, - 0, 0, 0, 0, 0, 0, 0, 13, 0, 4, - 5, 6, 7, 8, 9, 0, 3 -}; - -/* YYDEFGOTO[NTERM-NUM]. */ -static const yytype_int8 yydefgoto[] = -{ - -1, 5, 6 -}; - -/* YYPACT[STATE-NUM] -- Index in YYTABLE of the portion describing - STATE-NUM. */ -#define YYPACT_NINF -10 -static const yytype_int8 yypact[] = -{ - -9, -9, -10, -10, -9, 8, 36, -10, 13, -10, - -9, -9, -9, -9, -9, -9, -9, -10, 26, 41, - 45, 18, -2, 14, -10, -9, 36 -}; - -/* YYPGOTO[NTERM-NUM]. */ -static const yytype_int8 yypgoto[] = -{ - -10, -10, -1 -}; - -/* YYTABLE[YYPACT[STATE-NUM]]. What to do in state STATE-NUM. If - positive, shift that token. If negative, reduce the rule which - number is the opposite. If YYTABLE_NINF, syntax error. */ -#define YYTABLE_NINF -1 -static const yytype_uint8 yytable[] = -{ - 7, 1, 2, 8, 3, 4, 15, 16, 9, 18, - 19, 20, 21, 22, 23, 24, 10, 11, 12, 13, - 14, 15, 16, 16, 26, 14, 15, 16, 17, 10, - 11, 12, 13, 14, 15, 16, 0, 0, 25, 10, - 11, 12, 13, 14, 15, 16, 12, 13, 14, 15, - 16, 13, 14, 15, 16 -}; - -#define yypact_value_is_default(Yystate) \ - (!!((Yystate) == (-10))) - -#define yytable_value_is_error(Yytable_value) \ - YYID (0) - -static const yytype_int8 yycheck[] = -{ - 1, 10, 11, 4, 13, 14, 8, 9, 0, 10, - 11, 12, 13, 14, 15, 16, 3, 4, 5, 6, - 7, 8, 9, 9, 25, 7, 8, 9, 15, 3, - 4, 5, 6, 7, 8, 9, -1, -1, 12, 3, - 4, 5, 6, 7, 8, 9, 5, 6, 7, 8, - 9, 6, 7, 8, 9 -}; - -/* YYSTOS[STATE-NUM] -- The (internal number of the) accessing - symbol of state STATE-NUM. */ -static const yytype_uint8 yystos[] = -{ - 0, 10, 11, 13, 14, 17, 18, 18, 18, 0, - 3, 4, 5, 6, 7, 8, 9, 15, 18, 18, - 18, 18, 18, 18, 18, 12, 18 -}; - -#define yyerrok (yyerrstatus = 0) -#define yyclearin (yychar = YYEMPTY) -#define YYEMPTY (-2) -#define YYEOF 0 - -#define YYACCEPT goto yyacceptlab -#define YYABORT goto yyabortlab -#define YYERROR goto yyerrorlab - - -/* Like YYERROR except do call yyerror. This remains here temporarily - to ease the transition to the new meaning of YYERROR, for GCC. - Once GCC version 2 has supplanted version 1, this can go. However, - YYFAIL appears to be in use. Nevertheless, it is formally deprecated - in Bison 2.4.2's NEWS entry, where a plan to phase it out is - discussed. */ - -#define YYFAIL goto yyerrlab -#if defined YYFAIL - /* This is here to suppress warnings from the GCC cpp's - -Wunused-macros. Normally we don't worry about that warning, but - some users do, and we want to make it easy for users to remove - YYFAIL uses, which will produce warnings from Bison 2.5. */ -#endif - -#define YYRECOVERING() (!!yyerrstatus) - -#define YYBACKUP(Token, Value) \ -do \ - if (yychar == YYEMPTY) \ - { \ - yychar = (Token); \ - yylval = (Value); \ - YYPOPSTACK (yylen); \ - yystate = *yyssp; \ - goto yybackup; \ - } \ - else \ - { \ - yyerror (arg, YY_("syntax error: cannot back up")); \ - YYERROR; \ - } \ -while (YYID (0)) - -/* Error token number */ -#define YYTERROR 1 -#define YYERRCODE 256 - - -/* This macro is provided for backward compatibility. */ -#ifndef YY_LOCATION_PRINT -# define YY_LOCATION_PRINT(File, Loc) ((void) 0) -#endif - - -/* YYLEX -- calling `yylex' with the right arguments. */ -#ifdef YYLEX_PARAM -# define YYLEX yylex (&yylval, YYLEX_PARAM) -#else -# define YYLEX yylex (&yylval, arg) -#endif - -/* Enable debugging if requested. */ -#if YYDEBUG - -# ifndef YYFPRINTF -# include /* INFRINGES ON USER NAME SPACE */ -# define YYFPRINTF fprintf -# endif - -# define YYDPRINTF(Args) \ -do { \ - if (yydebug) \ - YYFPRINTF Args; \ -} while (YYID (0)) - -# define YY_SYMBOL_PRINT(Title, Type, Value, Location) \ -do { \ - if (yydebug) \ - { \ - YYFPRINTF (stderr, "%s ", Title); \ - yy_symbol_print (stderr, \ - Type, Value, arg); \ - YYFPRINTF (stderr, "\n"); \ - } \ -} while (YYID (0)) - - -/*--------------------------------. -| Print this symbol on YYOUTPUT. | -`--------------------------------*/ - -/*ARGSUSED*/ -#if (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -static void -yy_symbol_value_print (FILE *yyoutput, int yytype, YYSTYPE const * const yyvaluep, struct parse_args *arg) -#else -static void -yy_symbol_value_print (yyoutput, yytype, yyvaluep, arg) - FILE *yyoutput; - int yytype; - YYSTYPE const * const yyvaluep; - struct parse_args *arg; -#endif -{ - FILE *yyo = yyoutput; - YYUSE (yyo); - if (!yyvaluep) - return; - YYUSE (arg); -# ifdef YYPRINT - if (yytype < YYNTOKENS) - YYPRINT (yyoutput, yytoknum[yytype], *yyvaluep); -# else - YYUSE (yyoutput); -# endif - switch (yytype) - { - default: - break; - } -} - - -/*--------------------------------. -| Print this symbol on YYOUTPUT. | -`--------------------------------*/ - -#if (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -static void -yy_symbol_print (FILE *yyoutput, int yytype, YYSTYPE const * const yyvaluep, struct parse_args *arg) -#else -static void -yy_symbol_print (yyoutput, yytype, yyvaluep, arg) - FILE *yyoutput; - int yytype; - YYSTYPE const * const yyvaluep; - struct parse_args *arg; -#endif -{ - if (yytype < YYNTOKENS) - YYFPRINTF (yyoutput, "token %s (", yytname[yytype]); - else - YYFPRINTF (yyoutput, "nterm %s (", yytname[yytype]); - - yy_symbol_value_print (yyoutput, yytype, yyvaluep, arg); - YYFPRINTF (yyoutput, ")"); -} - -/*------------------------------------------------------------------. -| yy_stack_print -- Print the state stack from its BOTTOM up to its | -| TOP (included). | -`------------------------------------------------------------------*/ - -#if (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -static void -yy_stack_print (yytype_int16 *yybottom, yytype_int16 *yytop) -#else -static void -yy_stack_print (yybottom, yytop) - yytype_int16 *yybottom; - yytype_int16 *yytop; -#endif -{ - YYFPRINTF (stderr, "Stack now"); - for (; yybottom <= yytop; yybottom++) - { - int yybot = *yybottom; - YYFPRINTF (stderr, " %d", yybot); - } - YYFPRINTF (stderr, "\n"); -} - -# define YY_STACK_PRINT(Bottom, Top) \ -do { \ - if (yydebug) \ - yy_stack_print ((Bottom), (Top)); \ -} while (YYID (0)) - - -/*------------------------------------------------. -| Report that the YYRULE is going to be reduced. | -`------------------------------------------------*/ - -#if (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -static void -yy_reduce_print (YYSTYPE *yyvsp, int yyrule, struct parse_args *arg) -#else -static void -yy_reduce_print (yyvsp, yyrule, arg) - YYSTYPE *yyvsp; - int yyrule; - struct parse_args *arg; -#endif -{ - int yynrhs = yyr2[yyrule]; - int yyi; - unsigned long int yylno = yyrline[yyrule]; - YYFPRINTF (stderr, "Reducing stack by rule %d (line %lu):\n", - yyrule - 1, yylno); - /* The symbols being reduced. */ - for (yyi = 0; yyi < yynrhs; yyi++) - { - YYFPRINTF (stderr, " $%d = ", yyi + 1); - yy_symbol_print (stderr, yyrhs[yyprhs[yyrule] + yyi], - &(yyvsp[(yyi + 1) - (yynrhs)]) - , arg); - YYFPRINTF (stderr, "\n"); - } -} - -# define YY_REDUCE_PRINT(Rule) \ -do { \ - if (yydebug) \ - yy_reduce_print (yyvsp, Rule, arg); \ -} while (YYID (0)) - -/* Nonzero means print parse trace. It is left uninitialized so that - multiple parsers can coexist. */ -int yydebug; -#else /* !YYDEBUG */ -# define YYDPRINTF(Args) -# define YY_SYMBOL_PRINT(Title, Type, Value, Location) -# define YY_STACK_PRINT(Bottom, Top) -# define YY_REDUCE_PRINT(Rule) -#endif /* !YYDEBUG */ - - -/* YYINITDEPTH -- initial size of the parser's stacks. */ -#ifndef YYINITDEPTH -# define YYINITDEPTH 200 -#endif - -/* YYMAXDEPTH -- maximum size the stacks can grow to (effective only - if the built-in stack extension method is used). - - Do not make this value too large; the results are undefined if - YYSTACK_ALLOC_MAXIMUM < YYSTACK_BYTES (YYMAXDEPTH) - evaluated with infinite-precision integer arithmetic. */ - -#ifndef YYMAXDEPTH -# define YYMAXDEPTH 10000 -#endif - - -#if YYERROR_VERBOSE - -# ifndef yystrlen -# if defined __GLIBC__ && defined _STRING_H -# define yystrlen strlen -# else -/* Return the length of YYSTR. */ -#if (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -static YYSIZE_T -yystrlen (const char *yystr) -#else -static YYSIZE_T -yystrlen (yystr) - const char *yystr; -#endif -{ - YYSIZE_T yylen; - for (yylen = 0; yystr[yylen]; yylen++) - continue; - return yylen; -} -# endif -# endif - -# ifndef yystpcpy -# if defined __GLIBC__ && defined _STRING_H && defined _GNU_SOURCE -# define yystpcpy stpcpy -# else -/* Copy YYSRC to YYDEST, returning the address of the terminating '\0' in - YYDEST. */ -#if (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -static char * -yystpcpy (char *yydest, const char *yysrc) -#else -static char * -yystpcpy (yydest, yysrc) - char *yydest; - const char *yysrc; -#endif -{ - char *yyd = yydest; - const char *yys = yysrc; - - while ((*yyd++ = *yys++) != '\0') - continue; - - return yyd - 1; -} -# endif -# endif - -# ifndef yytnamerr -/* Copy to YYRES the contents of YYSTR after stripping away unnecessary - quotes and backslashes, so that it's suitable for yyerror. The - heuristic is that double-quoting is unnecessary unless the string - contains an apostrophe, a comma, or backslash (other than - backslash-backslash). YYSTR is taken from yytname. If YYRES is - null, do not copy; instead, return the length of what the result - would have been. */ -static YYSIZE_T -yytnamerr (char *yyres, const char *yystr) -{ - if (*yystr == '"') - { - YYSIZE_T yyn = 0; - char const *yyp = yystr; - - for (;;) - switch (*++yyp) - { - case '\'': - case ',': - goto do_not_strip_quotes; - - case '\\': - if (*++yyp != '\\') - goto do_not_strip_quotes; - /* Fall through. */ - default: - if (yyres) - yyres[yyn] = *yyp; - yyn++; - break; - - case '"': - if (yyres) - yyres[yyn] = '\0'; - return yyn; - } - do_not_strip_quotes: ; - } - - if (! yyres) - return yystrlen (yystr); - - return yystpcpy (yyres, yystr) - yyres; -} -# endif - -/* Copy into *YYMSG, which is of size *YYMSG_ALLOC, an error message - about the unexpected token YYTOKEN for the state stack whose top is - YYSSP. - - Return 0 if *YYMSG was successfully written. Return 1 if *YYMSG is - not large enough to hold the message. In that case, also set - *YYMSG_ALLOC to the required number of bytes. Return 2 if the - required number of bytes is too large to store. */ -static int -yysyntax_error (YYSIZE_T *yymsg_alloc, char **yymsg, - yytype_int16 *yyssp, int yytoken) -{ - YYSIZE_T yysize0 = yytnamerr (YY_NULL, yytname[yytoken]); - YYSIZE_T yysize = yysize0; - enum { YYERROR_VERBOSE_ARGS_MAXIMUM = 5 }; - /* Internationalized format string. */ - const char *yyformat = YY_NULL; - /* Arguments of yyformat. */ - char const *yyarg[YYERROR_VERBOSE_ARGS_MAXIMUM]; - /* Number of reported tokens (one for the "unexpected", one per - "expected"). */ - int yycount = 0; - - /* There are many possibilities here to consider: - - Assume YYFAIL is not used. It's too flawed to consider. See - - for details. YYERROR is fine as it does not invoke this - function. - - If this state is a consistent state with a default action, then - the only way this function was invoked is if the default action - is an error action. In that case, don't check for expected - tokens because there are none. - - The only way there can be no lookahead present (in yychar) is if - this state is a consistent state with a default action. Thus, - detecting the absence of a lookahead is sufficient to determine - that there is no unexpected or expected token to report. In that - case, just report a simple "syntax error". - - Don't assume there isn't a lookahead just because this state is a - consistent state with a default action. There might have been a - previous inconsistent state, consistent state with a non-default - action, or user semantic action that manipulated yychar. - - Of course, the expected token list depends on states to have - correct lookahead information, and it depends on the parser not - to perform extra reductions after fetching a lookahead from the - scanner and before detecting a syntax error. Thus, state merging - (from LALR or IELR) and default reductions corrupt the expected - token list. However, the list is correct for canonical LR with - one exception: it will still contain any token that will not be - accepted due to an error action in a later state. - */ - if (yytoken != YYEMPTY) - { - int yyn = yypact[*yyssp]; - yyarg[yycount++] = yytname[yytoken]; - if (!yypact_value_is_default (yyn)) - { - /* Start YYX at -YYN if negative to avoid negative indexes in - YYCHECK. In other words, skip the first -YYN actions for - this state because they are default actions. */ - int yyxbegin = yyn < 0 ? -yyn : 0; - /* Stay within bounds of both yycheck and yytname. */ - int yychecklim = YYLAST - yyn + 1; - int yyxend = yychecklim < YYNTOKENS ? yychecklim : YYNTOKENS; - int yyx; - - for (yyx = yyxbegin; yyx < yyxend; ++yyx) - if (yycheck[yyx + yyn] == yyx && yyx != YYTERROR - && !yytable_value_is_error (yytable[yyx + yyn])) - { - if (yycount == YYERROR_VERBOSE_ARGS_MAXIMUM) - { - yycount = 1; - yysize = yysize0; - break; - } - yyarg[yycount++] = yytname[yyx]; - { - YYSIZE_T yysize1 = yysize + yytnamerr (YY_NULL, yytname[yyx]); - if (! (yysize <= yysize1 - && yysize1 <= YYSTACK_ALLOC_MAXIMUM)) - return 2; - yysize = yysize1; - } - } - } - } - - switch (yycount) - { -# define YYCASE_(N, S) \ - case N: \ - yyformat = S; \ - break - YYCASE_(0, YY_("syntax error")); - YYCASE_(1, YY_("syntax error, unexpected %s")); - YYCASE_(2, YY_("syntax error, unexpected %s, expecting %s")); - YYCASE_(3, YY_("syntax error, unexpected %s, expecting %s or %s")); - YYCASE_(4, YY_("syntax error, unexpected %s, expecting %s or %s or %s")); - YYCASE_(5, YY_("syntax error, unexpected %s, expecting %s or %s or %s or %s")); -# undef YYCASE_ - } - - { - YYSIZE_T yysize1 = yysize + yystrlen (yyformat); - if (! (yysize <= yysize1 && yysize1 <= YYSTACK_ALLOC_MAXIMUM)) - return 2; - yysize = yysize1; - } - - if (*yymsg_alloc < yysize) - { - *yymsg_alloc = 2 * yysize; - if (! (yysize <= *yymsg_alloc - && *yymsg_alloc <= YYSTACK_ALLOC_MAXIMUM)) - *yymsg_alloc = YYSTACK_ALLOC_MAXIMUM; - return 1; - } - - /* Avoid sprintf, as that infringes on the user's name space. - Don't have undefined behavior even if the translation - produced a string with the wrong number of "%s"s. */ - { - char *yyp = *yymsg; - int yyi = 0; - while ((*yyp = *yyformat) != '\0') - if (*yyp == '%' && yyformat[1] == 's' && yyi < yycount) - { - yyp += yytnamerr (yyp, yyarg[yyi++]); - yyformat += 2; - } - else - { - yyp++; - yyformat++; - } - } - return 0; -} -#endif /* YYERROR_VERBOSE */ - -/*-----------------------------------------------. -| Release the memory associated to this symbol. | -`-----------------------------------------------*/ - -/*ARGSUSED*/ -#if (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -static void -yydestruct (const char *yymsg, int yytype, YYSTYPE *yyvaluep, struct parse_args *arg) -#else -static void -yydestruct (yymsg, yytype, yyvaluep, arg) - const char *yymsg; - int yytype; - YYSTYPE *yyvaluep; - struct parse_args *arg; -#endif -{ - YYUSE (yyvaluep); - YYUSE (arg); - - if (!yymsg) - yymsg = "Deleting"; - YY_SYMBOL_PRINT (yymsg, yytype, yyvaluep, yylocationp); - - switch (yytype) - { - - default: - break; - } -} - - - - -/*----------. -| yyparse. | -`----------*/ - -#ifdef YYPARSE_PARAM -#if (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -int -yyparse (void *YYPARSE_PARAM) -#else -int -yyparse (YYPARSE_PARAM) - void *YYPARSE_PARAM; -#endif -#else /* ! YYPARSE_PARAM */ -#if (defined __STDC__ || defined __C99__FUNC__ \ - || defined __cplusplus || defined _MSC_VER) -int -yyparse (struct parse_args *arg) -#else -int -yyparse (arg) - struct parse_args *arg; -#endif -#endif -{ -/* The lookahead symbol. */ -int yychar; - - -#if defined __GNUC__ && 407 <= __GNUC__ * 100 + __GNUC_MINOR__ -/* Suppress an incorrect diagnostic about yylval being uninitialized. */ -# define YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN \ - _Pragma ("GCC diagnostic push") \ - _Pragma ("GCC diagnostic ignored \"-Wuninitialized\"")\ - _Pragma ("GCC diagnostic ignored \"-Wmaybe-uninitialized\"") -# define YY_IGNORE_MAYBE_UNINITIALIZED_END \ - _Pragma ("GCC diagnostic pop") -#else -/* Default value used for initialization, for pacifying older GCCs - or non-GCC compilers. */ -static YYSTYPE yyval_default; -# define YY_INITIAL_VALUE(Value) = Value -#endif -#ifndef YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN -# define YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN -# define YY_IGNORE_MAYBE_UNINITIALIZED_END -#endif -#ifndef YY_INITIAL_VALUE -# define YY_INITIAL_VALUE(Value) /* Nothing. */ -#endif - -/* The semantic value of the lookahead symbol. */ -YYSTYPE yylval YY_INITIAL_VALUE(yyval_default); - - /* Number of syntax errors so far. */ - int yynerrs; - - int yystate; - /* Number of tokens to shift before error messages enabled. */ - int yyerrstatus; - - /* The stacks and their tools: - `yyss': related to states. - `yyvs': related to semantic values. - - Refer to the stacks through separate pointers, to allow yyoverflow - to reallocate them elsewhere. */ - - /* The state stack. */ - yytype_int16 yyssa[YYINITDEPTH]; - yytype_int16 *yyss; - yytype_int16 *yyssp; - - /* The semantic value stack. */ - YYSTYPE yyvsa[YYINITDEPTH]; - YYSTYPE *yyvs; - YYSTYPE *yyvsp; - - YYSIZE_T yystacksize; - - int yyn; - int yyresult; - /* Lookahead token as an internal (translated) token number. */ - int yytoken = 0; - /* The variables used to return semantic value and location from the - action routines. */ - YYSTYPE yyval; - -#if YYERROR_VERBOSE - /* Buffer for error messages, and its allocated size. */ - char yymsgbuf[128]; - char *yymsg = yymsgbuf; - YYSIZE_T yymsg_alloc = sizeof yymsgbuf; -#endif - -#define YYPOPSTACK(N) (yyvsp -= (N), yyssp -= (N)) - - /* The number of symbols on the RHS of the reduced rule. - Keep to zero when no symbol should be popped. */ - int yylen = 0; - - yyssp = yyss = yyssa; - yyvsp = yyvs = yyvsa; - yystacksize = YYINITDEPTH; - - YYDPRINTF ((stderr, "Starting parse\n")); - - yystate = 0; - yyerrstatus = 0; - yynerrs = 0; - yychar = YYEMPTY; /* Cause a token to be read. */ - goto yysetstate; - -/*------------------------------------------------------------. -| yynewstate -- Push a new state, which is found in yystate. | -`------------------------------------------------------------*/ - yynewstate: - /* In all cases, when you get here, the value and location stacks - have just been pushed. So pushing a state here evens the stacks. */ - yyssp++; - - yysetstate: - *yyssp = yystate; - - if (yyss + yystacksize - 1 <= yyssp) - { - /* Get the current used size of the three stacks, in elements. */ - YYSIZE_T yysize = yyssp - yyss + 1; - -#ifdef yyoverflow - { - /* Give user a chance to reallocate the stack. Use copies of - these so that the &'s don't force the real ones into - memory. */ - YYSTYPE *yyvs1 = yyvs; - yytype_int16 *yyss1 = yyss; - - /* Each stack pointer address is followed by the size of the - data in use in that stack, in bytes. This used to be a - conditional around just the two extra args, but that might - be undefined if yyoverflow is a macro. */ - yyoverflow (YY_("memory exhausted"), - &yyss1, yysize * sizeof (*yyssp), - &yyvs1, yysize * sizeof (*yyvsp), - &yystacksize); - - yyss = yyss1; - yyvs = yyvs1; - } -#else /* no yyoverflow */ -# ifndef YYSTACK_RELOCATE - goto yyexhaustedlab; -# else - /* Extend the stack our own way. */ - if (YYMAXDEPTH <= yystacksize) - goto yyexhaustedlab; - yystacksize *= 2; - if (YYMAXDEPTH < yystacksize) - yystacksize = YYMAXDEPTH; - - { - yytype_int16 *yyss1 = yyss; - union yyalloc *yyptr = - (union yyalloc *) YYSTACK_ALLOC (YYSTACK_BYTES (yystacksize)); - if (! yyptr) - goto yyexhaustedlab; - YYSTACK_RELOCATE (yyss_alloc, yyss); - YYSTACK_RELOCATE (yyvs_alloc, yyvs); -# undef YYSTACK_RELOCATE - if (yyss1 != yyssa) - YYSTACK_FREE (yyss1); - } -# endif -#endif /* no yyoverflow */ - - yyssp = yyss + yysize - 1; - yyvsp = yyvs + yysize - 1; - - YYDPRINTF ((stderr, "Stack size increased to %lu\n", - (unsigned long int) yystacksize)); - - if (yyss + yystacksize - 1 <= yyssp) - YYABORT; - } - - YYDPRINTF ((stderr, "Entering state %d\n", yystate)); - - if (yystate == YYFINAL) - YYACCEPT; - - goto yybackup; - -/*-----------. -| yybackup. | -`-----------*/ -yybackup: - - /* Do appropriate processing given the current state. Read a - lookahead token if we need one and don't already have one. */ - - /* First try to decide what to do without reference to lookahead token. */ - yyn = yypact[yystate]; - if (yypact_value_is_default (yyn)) - goto yydefault; - - /* Not known => get a lookahead token if don't already have one. */ - - /* YYCHAR is either YYEMPTY or YYEOF or a valid lookahead symbol. */ - if (yychar == YYEMPTY) - { - YYDPRINTF ((stderr, "Reading a token: ")); - yychar = YYLEX; - } - - if (yychar <= YYEOF) - { - yychar = yytoken = YYEOF; - YYDPRINTF ((stderr, "Now at end of input.\n")); - } - else - { - yytoken = YYTRANSLATE (yychar); - YY_SYMBOL_PRINT ("Next token is", yytoken, &yylval, &yylloc); - } - - /* If the proper action on seeing token YYTOKEN is to reduce or to - detect an error, take that action. */ - yyn += yytoken; - if (yyn < 0 || YYLAST < yyn || yycheck[yyn] != yytoken) - goto yydefault; - yyn = yytable[yyn]; - if (yyn <= 0) - { - if (yytable_value_is_error (yyn)) - goto yyerrlab; - yyn = -yyn; - goto yyreduce; - } - - /* Count tokens shifted since error; after three, turn off error - status. */ - if (yyerrstatus) - yyerrstatus--; - - /* Shift the lookahead token. */ - YY_SYMBOL_PRINT ("Shifting", yytoken, &yylval, &yylloc); - - /* Discard the shifted token. */ - yychar = YYEMPTY; - - yystate = yyn; - YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN - *++yyvsp = yylval; - YY_IGNORE_MAYBE_UNINITIALIZED_END - - goto yynewstate; - - -/*-----------------------------------------------------------. -| yydefault -- do the default action for the current state. | -`-----------------------------------------------------------*/ -yydefault: - yyn = yydefact[yystate]; - if (yyn == 0) - goto yyerrlab; - goto yyreduce; - - -/*-----------------------------. -| yyreduce -- Do a reduction. | -`-----------------------------*/ -yyreduce: - /* yyn is the number of a rule to reduce with. */ - yylen = yyr2[yyn]; - - /* If YYLEN is nonzero, implement the default value of the action: - `$$ = $1'. - - Otherwise, the following line sets YYVAL to garbage. - This behavior is undocumented and Bison - users should not rely upon it. Assigning to YYVAL - unconditionally makes the parser a bit smaller, and it avoids a - GCC warning that YYVAL may be used uninitialized. */ - yyval = yyvsp[1-yylen]; - - - YY_REDUCE_PRINT (yyn); - switch (yyn) - { - case 2: -/* Line 1792 of yacc.c */ -#line 153 "plural.y" - { - if ((yyvsp[(1) - (1)].exp) == NULL) - YYABORT; - arg->res = (yyvsp[(1) - (1)].exp); - } - break; - - case 3: -/* Line 1792 of yacc.c */ -#line 161 "plural.y" - { - (yyval.exp) = new_exp_3 (qmop, (yyvsp[(1) - (5)].exp), (yyvsp[(3) - (5)].exp), (yyvsp[(5) - (5)].exp)); - } - break; - - case 4: -/* Line 1792 of yacc.c */ -#line 165 "plural.y" - { - (yyval.exp) = new_exp_2 (lor, (yyvsp[(1) - (3)].exp), (yyvsp[(3) - (3)].exp)); - } - break; - - case 5: -/* Line 1792 of yacc.c */ -#line 169 "plural.y" - { - (yyval.exp) = new_exp_2 (land, (yyvsp[(1) - (3)].exp), (yyvsp[(3) - (3)].exp)); - } - break; - - case 6: -/* Line 1792 of yacc.c */ -#line 173 "plural.y" - { - (yyval.exp) = new_exp_2 ((yyvsp[(2) - (3)].op), (yyvsp[(1) - (3)].exp), (yyvsp[(3) - (3)].exp)); - } - break; - - case 7: -/* Line 1792 of yacc.c */ -#line 177 "plural.y" - { - (yyval.exp) = new_exp_2 ((yyvsp[(2) - (3)].op), (yyvsp[(1) - (3)].exp), (yyvsp[(3) - (3)].exp)); - } - break; - - case 8: -/* Line 1792 of yacc.c */ -#line 181 "plural.y" - { - (yyval.exp) = new_exp_2 ((yyvsp[(2) - (3)].op), (yyvsp[(1) - (3)].exp), (yyvsp[(3) - (3)].exp)); - } - break; - - case 9: -/* Line 1792 of yacc.c */ -#line 185 "plural.y" - { - (yyval.exp) = new_exp_2 ((yyvsp[(2) - (3)].op), (yyvsp[(1) - (3)].exp), (yyvsp[(3) - (3)].exp)); - } - break; - - case 10: -/* Line 1792 of yacc.c */ -#line 189 "plural.y" - { - (yyval.exp) = new_exp_1 (lnot, (yyvsp[(2) - (2)].exp)); - } - break; - - case 11: -/* Line 1792 of yacc.c */ -#line 193 "plural.y" - { - (yyval.exp) = new_exp_0 (var); - } - break; - - case 12: -/* Line 1792 of yacc.c */ -#line 197 "plural.y" - { - if (((yyval.exp) = new_exp_0 (num)) != NULL) - (yyval.exp)->val.num = (yyvsp[(1) - (1)].num); - } - break; - - case 13: -/* Line 1792 of yacc.c */ -#line 202 "plural.y" - { - (yyval.exp) = (yyvsp[(2) - (3)].exp); - } - break; - - -/* Line 1792 of yacc.c */ -#line 1604 "plural.c" - default: break; - } - /* User semantic actions sometimes alter yychar, and that requires - that yytoken be updated with the new translation. We take the - approach of translating immediately before every use of yytoken. - One alternative is translating here after every semantic action, - but that translation would be missed if the semantic action invokes - YYABORT, YYACCEPT, or YYERROR immediately after altering yychar or - if it invokes YYBACKUP. In the case of YYABORT or YYACCEPT, an - incorrect destructor might then be invoked immediately. In the - case of YYERROR or YYBACKUP, subsequent parser actions might lead - to an incorrect destructor call or verbose syntax error message - before the lookahead is translated. */ - YY_SYMBOL_PRINT ("-> $$ =", yyr1[yyn], &yyval, &yyloc); - - YYPOPSTACK (yylen); - yylen = 0; - YY_STACK_PRINT (yyss, yyssp); - - *++yyvsp = yyval; - - /* Now `shift' the result of the reduction. Determine what state - that goes to, based on the state we popped back to and the rule - number reduced by. */ - - yyn = yyr1[yyn]; - - yystate = yypgoto[yyn - YYNTOKENS] + *yyssp; - if (0 <= yystate && yystate <= YYLAST && yycheck[yystate] == *yyssp) - yystate = yytable[yystate]; - else - yystate = yydefgoto[yyn - YYNTOKENS]; - - goto yynewstate; - - -/*------------------------------------. -| yyerrlab -- here on detecting error | -`------------------------------------*/ -yyerrlab: - /* Make sure we have latest lookahead translation. See comments at - user semantic actions for why this is necessary. */ - yytoken = yychar == YYEMPTY ? YYEMPTY : YYTRANSLATE (yychar); - - /* If not already recovering from an error, report this error. */ - if (!yyerrstatus) - { - ++yynerrs; -#if ! YYERROR_VERBOSE - yyerror (arg, YY_("syntax error")); -#else -# define YYSYNTAX_ERROR yysyntax_error (&yymsg_alloc, &yymsg, \ - yyssp, yytoken) - { - char const *yymsgp = YY_("syntax error"); - int yysyntax_error_status; - yysyntax_error_status = YYSYNTAX_ERROR; - if (yysyntax_error_status == 0) - yymsgp = yymsg; - else if (yysyntax_error_status == 1) - { - if (yymsg != yymsgbuf) - YYSTACK_FREE (yymsg); - yymsg = (char *) YYSTACK_ALLOC (yymsg_alloc); - if (!yymsg) - { - yymsg = yymsgbuf; - yymsg_alloc = sizeof yymsgbuf; - yysyntax_error_status = 2; - } - else - { - yysyntax_error_status = YYSYNTAX_ERROR; - yymsgp = yymsg; - } - } - yyerror (arg, yymsgp); - if (yysyntax_error_status == 2) - goto yyexhaustedlab; - } -# undef YYSYNTAX_ERROR -#endif - } - - - - if (yyerrstatus == 3) - { - /* If just tried and failed to reuse lookahead token after an - error, discard it. */ - - if (yychar <= YYEOF) - { - /* Return failure if at end of input. */ - if (yychar == YYEOF) - YYABORT; - } - else - { - yydestruct ("Error: discarding", - yytoken, &yylval, arg); - yychar = YYEMPTY; - } - } - - /* Else will try to reuse lookahead token after shifting the error - token. */ - goto yyerrlab1; - - -/*---------------------------------------------------. -| yyerrorlab -- error raised explicitly by YYERROR. | -`---------------------------------------------------*/ -yyerrorlab: - - /* Pacify compilers like GCC when the user code never invokes - YYERROR and the label yyerrorlab therefore never appears in user - code. */ - if (/*CONSTCOND*/ 0) - goto yyerrorlab; - - /* Do not reclaim the symbols of the rule which action triggered - this YYERROR. */ - YYPOPSTACK (yylen); - yylen = 0; - YY_STACK_PRINT (yyss, yyssp); - yystate = *yyssp; - goto yyerrlab1; - - -/*-------------------------------------------------------------. -| yyerrlab1 -- common code for both syntax error and YYERROR. | -`-------------------------------------------------------------*/ -yyerrlab1: - yyerrstatus = 3; /* Each real token shifted decrements this. */ - - for (;;) - { - yyn = yypact[yystate]; - if (!yypact_value_is_default (yyn)) - { - yyn += YYTERROR; - if (0 <= yyn && yyn <= YYLAST && yycheck[yyn] == YYTERROR) - { - yyn = yytable[yyn]; - if (0 < yyn) - break; - } - } - - /* Pop the current state because it cannot handle the error token. */ - if (yyssp == yyss) - YYABORT; - - - yydestruct ("Error: popping", - yystos[yystate], yyvsp, arg); - YYPOPSTACK (1); - yystate = *yyssp; - YY_STACK_PRINT (yyss, yyssp); - } - - YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN - *++yyvsp = yylval; - YY_IGNORE_MAYBE_UNINITIALIZED_END - - - /* Shift the error token. */ - YY_SYMBOL_PRINT ("Shifting", yystos[yyn], yyvsp, yylsp); - - yystate = yyn; - goto yynewstate; - - -/*-------------------------------------. -| yyacceptlab -- YYACCEPT comes here. | -`-------------------------------------*/ -yyacceptlab: - yyresult = 0; - goto yyreturn; - -/*-----------------------------------. -| yyabortlab -- YYABORT comes here. | -`-----------------------------------*/ -yyabortlab: - yyresult = 1; - goto yyreturn; - -#if !defined yyoverflow || YYERROR_VERBOSE -/*-------------------------------------------------. -| yyexhaustedlab -- memory exhaustion comes here. | -`-------------------------------------------------*/ -yyexhaustedlab: - yyerror (arg, YY_("memory exhausted")); - yyresult = 2; - /* Fall through. */ -#endif - -yyreturn: - if (yychar != YYEMPTY) - { - /* Make sure we have latest lookahead translation. See comments at - user semantic actions for why this is necessary. */ - yytoken = YYTRANSLATE (yychar); - yydestruct ("Cleanup: discarding lookahead", - yytoken, &yylval, arg); - } - /* Do not reclaim the symbols of the rule which action triggered - this YYABORT or YYACCEPT. */ - YYPOPSTACK (yylen); - YY_STACK_PRINT (yyss, yyssp); - while (yyssp != yyss) - { - yydestruct ("Cleanup: popping", - yystos[*yyssp], yyvsp, arg); - YYPOPSTACK (1); - } -#ifndef yyoverflow - if (yyss != yyssa) - YYSTACK_FREE (yyss); -#endif -#if YYERROR_VERBOSE - if (yymsg != yymsgbuf) - YYSTACK_FREE (yymsg); -#endif - /* Make sure YYID is used. */ - return YYID (yyresult); -} - - -/* Line 2055 of yacc.c */ -#line 207 "plural.y" - - -void -FREE_EXPRESSION (struct expression *exp) -{ - if (exp == NULL) - return; - - /* Handle the recursive case. */ - switch (exp->nargs) - { - case 3: - FREE_EXPRESSION (exp->val.args[2]); - /* FALLTHROUGH */ - case 2: - FREE_EXPRESSION (exp->val.args[1]); - /* FALLTHROUGH */ - case 1: - FREE_EXPRESSION (exp->val.args[0]); - /* FALLTHROUGH */ - default: - break; - } - - free (exp); -} - - -static int -yylex (YYSTYPE *lval, struct parse_args *arg) -{ - const char *exp = arg->cp; - int result; - - while (1) - { - if (exp[0] == '\0') - { - arg->cp = exp; - return YYEOF; - } - - if (exp[0] != ' ' && exp[0] != '\t') - break; - - ++exp; - } - - result = *exp++; - switch (result) - { - case '0': case '1': case '2': case '3': case '4': - case '5': case '6': case '7': case '8': case '9': - { - unsigned long int n = result - '0'; - while (exp[0] >= '0' && exp[0] <= '9') - { - n *= 10; - n += exp[0] - '0'; - ++exp; - } - lval->num = n; - result = NUMBER; - } - break; - - case '=': - if (exp[0] == '=') - { - ++exp; - lval->op = equal; - result = EQUOP2; - } - else - result = YYERRCODE; - break; - - case '!': - if (exp[0] == '=') - { - ++exp; - lval->op = not_equal; - result = EQUOP2; - } - break; - - case '&': - case '|': - if (exp[0] == result) - ++exp; - else - result = YYERRCODE; - break; - - case '<': - if (exp[0] == '=') - { - ++exp; - lval->op = less_or_equal; - } - else - lval->op = less_than; - result = CMPOP2; - break; - - case '>': - if (exp[0] == '=') - { - ++exp; - lval->op = greater_or_equal; - } - else - lval->op = greater_than; - result = CMPOP2; - break; - - case '*': - lval->op = mult; - result = MULOP2; - break; - - case '/': - lval->op = divide; - result = MULOP2; - break; - - case '%': - lval->op = module; - result = MULOP2; - break; - - case '+': - lval->op = plus; - result = ADDOP2; - break; - - case '-': - lval->op = minus; - result = ADDOP2; - break; - - case 'n': - case '?': - case ':': - case '(': - case ')': - /* Nothing, just return the character. */ - break; - - case ';': - case '\n': - case '\0': - /* Be safe and let the user call this function again. */ - --exp; - result = YYEOF; - break; - - default: - result = YYERRCODE; -#if YYDEBUG != 0 - --exp; -#endif - break; - } - - arg->cp = exp; - - return result; -} - - -static void -yyerror (struct parse_args *arg, const char *str) -{ - /* Do nothing. We don't print error messages here. */ -} diff --git a/manual/install.texi b/manual/install.texi index f1fa28c937..ea559e4e72 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -509,6 +509,12 @@ function, which was introduced in version 3.1.2 of @code{gawk}. As of release time, @code{gawk} version 4.1.3 is the newest verified to work to build @theglibc{}. +@item +GNU @code{bison} 2.7 or later + +@code{bison} is used to generate the @code{yacc} parser code in the @file{intl} +subdirectory. + @item Perl 5 @@ -540,14 +546,6 @@ and if you change any of the message translation files you will need GNU @code{gettext} 0.10.36 or later @end itemize -@noindent -If you wish to regenerate the @code{yacc} parser code in the @file{intl} -subdirectory you will need - -@itemize @bullet -@item -GNU @code{bison} 2.7 or later -@end itemize @noindent You may also need these packages if you upgrade your source tree using -- cgit v1.2.3 From 07ed18d26a342741cb25a4739158c65ed9dd4d09 Mon Sep 17 00:00:00 2001 From: Rogerio Alves Date: Tue, 5 Dec 2017 14:24:14 -0200 Subject: Add elision tunables This patch adds several new tunables to control the behavior of elision on supported platforms[1]. Since elision now depends on tunables, we should always *compile* with elision enabled, and leave the code disabled, but available for runtime selection. This gives us *much* better compile-time testing of the existing code to avoid bit-rot[2]. Tested on ppc, ppc64, ppc64le, s390x and x86_64. [1] This part of the patch was initially proposed by Paul Murphy but was "staled" because the framework have changed since the patch was originally proposed: https://patchwork.sourceware.org/patch/10342/ [2] This part of the patch was inititally proposed as a RFC by Carlos O'Donnell. Make sense to me integrate this on the patch: https://sourceware.org/ml/libc-alpha/2017-05/msg00335.html * elf/dl-tunables.list: Add elision parameters. * manual/tunables.texi: Add entries about elision tunable. * sysdeps/unix/sysv/linux/powerpc/elision-conf.c: Add callback functions to dynamically enable/disable elision. Add multiple callbacks functions to set elision parameters. Deleted __libc_enable_secure check. * sysdeps/unix/sysv/linux/s390/elision-conf.c: Likewise. * sysdeps/unix/sysv/linux/x86/elision-conf.c: Likewise. * configure: Regenerated. * configure.ac: Option enable_lock_elision was deleted. * config.h.in: ENABLE_LOCK_ELISION flag was deleted. * config.make.in: Remove references to enable_lock_elision. * manual/install.texi: Elision configure option was removed. * INSTALL: Regenerated to remove enable_lock_elision. * nptl/Makefile: Disable elision so it can verify error case for destroying a mutex. * sysdeps/powerpc/nptl/elide.h: Cleanup ENABLE_LOCK_ELISION check. Deleted macros for the case when ENABLE_LOCK_ELISION was not defined. * sysdeps/s390/configure: Regenerated. * sysdeps/s390/configure.ac: Remove references to enable_lock_elision.. * nptl/tst-mutex8.c: Deleted all #ifndef ENABLE_LOCK_ELISION from the test. * sysdeps/powerpc/powerpc32/sysdep.h: Deleted all ENABLE_LOCK_ELISION checks. * sysdeps/powerpc/powerpc64/sysdep.h: Likewise. * sysdeps/powerpc/sysdep.h: Likewise. * sysdeps/s390/nptl/bits/pthreadtypes-arch.h: Likewise. * sysdeps/unix/sysv/linux/powerpc/force-elision.h: Likewise. * sysdeps/unix/sysv/linux/s390/elision-conf.h: Likewise. * sysdeps/unix/sysv/linux/s390/force-elision.h: Likewise. * sysdeps/unix/sysv/linux/s390/lowlevellock.h: Likewise. * sysdeps/unix/sysv/linux/s390/Makefile: Remove references to enable-lock-elision. Reviewed-by: Tulio Magno Quites Machado Filho --- ChangeLog | 39 +++++++++++++ INSTALL | 3 - config.h.in | 3 - config.make.in | 1 - configure | 17 ------ configure.ac | 10 ---- elf/dl-tunables.list | 34 +++++++++++ manual/install.texi | 3 - manual/tunables.texi | 69 +++++++++++++++++++++++ nptl/Makefile | 4 ++ nptl/tst-mutex8.c | 12 +--- sysdeps/powerpc/nptl/elide.h | 9 --- sysdeps/powerpc/powerpc32/sysdep.h | 2 +- sysdeps/powerpc/powerpc64/sysdep.h | 2 +- sysdeps/powerpc/sysdep.h | 4 +- sysdeps/s390/configure | 2 +- sysdeps/s390/configure.ac | 2 +- sysdeps/s390/nptl/bits/pthreadtypes-arch.h | 4 -- sysdeps/unix/sysv/linux/powerpc/elision-conf.c | 75 +++++++++++++++++++++++-- sysdeps/unix/sysv/linux/powerpc/force-elision.h | 2 - sysdeps/unix/sysv/linux/s390/Makefile | 2 - sysdeps/unix/sysv/linux/s390/elision-conf.c | 72 ++++++++++++++++++++++-- sysdeps/unix/sysv/linux/s390/elision-conf.h | 2 - sysdeps/unix/sysv/linux/s390/force-elision.h | 2 - sysdeps/unix/sysv/linux/s390/lowlevellock.h | 2 - sysdeps/unix/sysv/linux/x86/elision-conf.c | 74 +++++++++++++++++++++--- 26 files changed, 357 insertions(+), 94 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index c2a78bbf40..98ae6ac712 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,42 @@ +2017-12-05 Rogerio A. Cardoso , + Paul E. Murphy , + Carlos O'Donnell + + * elf/dl-tunables.list: Add elision parameters. + * manual/tunables.texi: Add entries about elision tunable. + * sysdeps/unix/sysv/linux/powerpc/elision-conf.c: + Add callback functions to dynamically enable/disable elision. + Add multiple callbacks functions to set elision parameters. + Deleted __libc_enable_secure check. + * sysdeps/unix/sysv/linux/s390/elision-conf.c: Likewise. + * sysdeps/unix/sysv/linux/x86/elision-conf.c: Likewise. + * configure: Regenerated. + * configure.ac: Option enable_lock_elision was deleted. + * config.h.in: ENABLE_LOCK_ELISION flag was deleted. + * config.make.in: Remove references to enable_lock_elision. + * manual/install.texi: Elision configure option was removed. + * INSTALL: Regenerated to remove enable_lock_elision. + * nptl/Makefile: + Disable elision so it can verify error case for destroying a mutex. + * sysdeps/powerpc/nptl/elide.h: + Cleanup ENABLE_LOCK_ELISION check. + Deleted macros for the case when ENABLE_LOCK_ELISION was not defined. + * sysdeps/s390/configure: Regenerated. + * sysdeps/s390/configure.ac: Remove references to enable_lock_elision.. + * nptl/tst-mutex8.c: + Deleted all #ifndef ENABLE_LOCK_ELISION from the test. + * sysdeps/powerpc/powerpc32/sysdep.h: + Deleted all ENABLE_LOCK_ELISION checks. + * sysdeps/powerpc/powerpc64/sysdep.h: Likewise. + * sysdeps/powerpc/sysdep.h: Likewise. + * sysdeps/s390/nptl/bits/pthreadtypes-arch.h: Likewise. + * sysdeps/unix/sysv/linux/powerpc/force-elision.h: Likewise. + * sysdeps/unix/sysv/linux/s390/elision-conf.h: Likewise. + * sysdeps/unix/sysv/linux/s390/force-elision.h: Likewise. + * sysdeps/unix/sysv/linux/s390/lowlevellock.h: Likewise. + * sysdeps/unix/sysv/linux/s390/Makefile: Remove references to + enable-lock-elision. + 2017-12-05 Joseph Myers * stdlib/strtod.c: Include . diff --git a/INSTALL b/INSTALL index d1a34c2a90..e59c11dabd 100644 --- a/INSTALL +++ b/INSTALL @@ -116,9 +116,6 @@ will be used, and CFLAGS sets optimization options for the compiler. formats may change over time. Consult the 'timezone' subdirectory for more details. -'--enable-lock-elision=yes' - Enable lock elision for pthread mutexes by default. - '--enable-stack-protector' '--enable-stack-protector=strong' '--enable-stack-protector=all' diff --git a/config.h.in b/config.h.in index 8d76dadca2..3c91d597ff 100644 --- a/config.h.in +++ b/config.h.in @@ -131,9 +131,6 @@ /* Define if __stack_chk_guard canary should be randomized at program startup. */ #undef ENABLE_STACKGUARD_RANDOMIZE -/* Define if lock elision should be enabled by default. */ -#undef ENABLE_LOCK_ELISION - /* Package description. */ #undef PKGVERSION diff --git a/config.make.in b/config.make.in index bd84a5747d..9da77d1efa 100644 --- a/config.make.in +++ b/config.make.in @@ -99,7 +99,6 @@ build-nscd = @build_nscd@ use-nscd = @use_nscd@ build-hardcoded-path-in-tests= @hardcoded_path_in_tests@ build-pt-chown = @build_pt_chown@ -enable-lock-elision = @enable_lock_elision@ have-tunables = @have_tunables@ # Build tools. diff --git a/configure b/configure index caba102846..dd8b8c95e6 100755 --- a/configure +++ b/configure @@ -679,7 +679,6 @@ enable_werror all_warnings force_install bindnow -enable_lock_elision hardcoded_path_in_tests enable_timezone_tools use_default_link @@ -768,7 +767,6 @@ enable_profile enable_timezone_tools enable_hardcoded_path_in_tests enable_stackguard_randomization -enable_lock_elision enable_hidden_plt enable_bind_now enable_stack_protector @@ -1428,8 +1426,6 @@ Optional Features: --enable-stackguard-randomization initialize __stack_chk_guard canary with a random number at program start - --enable-lock-elision=yes/no - Enable lock elision for pthread mutexes by default --disable-hidden-plt do not hide internal function calls to avoid PLT --enable-bind-now disable lazy relocations in DSOs --enable-stack-protector=[yes|no|all|strong] @@ -3395,19 +3391,6 @@ if test "$enable_stackguard_randomize" = yes; then fi -# Check whether --enable-lock-elision was given. -if test "${enable_lock_elision+set}" = set; then : - enableval=$enable_lock_elision; enable_lock_elision=$enableval -else - enable_lock_elision=no -fi - - -if test "$enable_lock_elision" = yes ; then - $as_echo "#define ENABLE_LOCK_ELISION 1" >>confdefs.h - -fi - # Check whether --enable-hidden-plt was given. if test "${enable_hidden_plt+set}" = set; then : enableval=$enable_hidden_plt; hidden=$enableval diff --git a/configure.ac b/configure.ac index 9707ae4d8e..f85a50da53 100644 --- a/configure.ac +++ b/configure.ac @@ -199,16 +199,6 @@ if test "$enable_stackguard_randomize" = yes; then AC_DEFINE(ENABLE_STACKGUARD_RANDOMIZE) fi -AC_ARG_ENABLE([lock-elision], - AC_HELP_STRING([--enable-lock-elision[=yes/no]], - [Enable lock elision for pthread mutexes by default]), - [enable_lock_elision=$enableval], - [enable_lock_elision=no]) -AC_SUBST(enable_lock_elision) -if test "$enable_lock_elision" = yes ; then - AC_DEFINE(ENABLE_LOCK_ELISION) -fi - AC_ARG_ENABLE([hidden-plt], AC_HELP_STRING([--disable-hidden-plt], [do not hide internal function calls to avoid PLT]), diff --git a/elf/dl-tunables.list b/elf/dl-tunables.list index c188c6ad52..ec0fe20787 100644 --- a/elf/dl-tunables.list +++ b/elf/dl-tunables.list @@ -96,4 +96,38 @@ glibc { default: HWCAP_IMPORTANT } } + + elision { + enable { + type: INT_32 + minval: 0 + maxval: 1 + security_level: SXID_ERASE + } + skip_lock_busy { + type: INT_32 + default: 3 + security_level: SXID_ERASE + } + skip_lock_internal_abort { + type: INT_32 + default: 3 + security_level: SXID_ERASE + } + skip_lock_after_retries { + type: INT_32 + default: 3 + security_level: SXID_ERASE + } + tries { + type: INT_32 + default: 3 + security_level: SXID_ERASE + } + skip_trylock_internal_abort { + type: INT_32 + default: 3 + security_level: SXID_ERASE + } + } } diff --git a/manual/install.texi b/manual/install.texi index ea559e4e72..a3cb09dafc 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -145,9 +145,6 @@ Note that you need to make sure the external tools are kept in sync with the versions that @theglibc{} expects as the data formats may change over time. Consult the @file{timezone} subdirectory for more details. -@item --enable-lock-elision=yes -Enable lock elision for pthread mutexes by default. - @item --enable-stack-protector @itemx --enable-stack-protector=strong @itemx --enable-stack-protector=all diff --git a/manual/tunables.texi b/manual/tunables.texi index f503daef56..e851b95c59 100644 --- a/manual/tunables.texi +++ b/manual/tunables.texi @@ -31,6 +31,7 @@ their own namespace. @menu * Tunable names:: The structure of a tunable name * Memory Allocation Tunables:: Tunables in the memory allocation subsystem +* Elision Tunables:: Tunables in elision subsystem * Hardware Capability Tunables:: Tunables that modify the hardware capabilities seen by @theglibc{} @end menu @@ -212,6 +213,74 @@ pre-fill the per-thread cache with. The default, or when set to zero, is no limit. @end deftp +@node Elision Tunables +@section Elision Tunables +@cindex elision tunables +@cindex tunables, elision + +@deftp {Tunable namespace} glibc.elision +Contended locks are usually slow and can lead to performance and scalability +issues in multithread code. Lock elision will use memory transactions to under +certain conditions, to elide locks and improve performance. +Elision behavior can be modified by setting the following tunables in +the @code{elision} namespace: +@end deftp + +@deftp Tunable glibc.elision.enable +The @code{glibc.elision.enable} tunable enables lock elision if the feature is +supported by the hardware. If elision is not supported by the hardware this +tunable has no effect. + +Elision tunables are supported for 64-bit Intel, IBM POWER, and z System +architectures. +@end deftp + +@deftp Tunable glibc.elision.skip_lock_busy +The @code{glibc.elision.skip_lock_busy} tunable sets how many times to use a +non-transactional lock after a transactional failure has occurred because the +lock is already acquired. Expressed in number of lock acquisition attempts. + +The default value of this tunable is @samp{3}. +@end deftp + +@deftp Tunable glibc.elision.skip_lock_internal_abort +The @code{glibc.elision.skip_lock_internal_abort} tunable sets how many times +the thread should avoid using elision if a transaction aborted for any reason +other than a different thread's memory accesses. Expressed in number of lock +acquisition attempts. + +The default value of this tunable is @samp{3}. +@end deftp + +@deftp Tunable glibc.elision.skip_lock_after_retries +The @code{glibc.elision.skip_lock_after_retries} tunable sets how many times +to try to elide a lock with transactions, that only failed due to a different +thread's memory accesses, before falling back to regular lock. +Expressed in number of lock elision attempts. + +This tunable is supported only on IBM POWER, and z System architectures. + +The default value of this tunable is @samp{3}. +@end deftp + +@deftp Tunable glibc.elision.tries +The @code{glibc.elision.tries} sets how many times to retry elision if there is +chance for the transaction to finish execution e.g., it wasn't +aborted due to the lock being already acquired. If elision is not supported +by the hardware this tunable is set to @samp{0} to avoid retries. + +The default value of this tunable is @samp{3}. +@end deftp + +@deftp Tunable glibc.elision.skip_trylock_internal_abort +The @code{glibc.elision.skip_trylock_internal_abort} tunable sets how many +times the thread should avoid trying the lock if a transaction aborted due to +reasons other than a different thread's memory accesses. Expressed in number +of try lock attempts. + +The default value of this tunable is @samp{3}. +@end deftp + @node Hardware Capability Tunables @section Hardware Capability Tunables @cindex hardware capability tunables diff --git a/nptl/Makefile b/nptl/Makefile index b0215e12d7..11e6ecd88b 100644 --- a/nptl/Makefile +++ b/nptl/Makefile @@ -714,6 +714,10 @@ endif $(objpfx)tst-compat-forwarder: $(objpfx)tst-compat-forwarder-mod.so +# Disable elision for tst-mutex8 so it can verify error case for +# destroying a mutex. +tst-mutex8-ENV = GLIBC_TUNABLES=glibc.elision.enable=0 + # The tests here better do not run in parallel ifneq ($(filter %tests,$(MAKECMDGOALS)),) .NOTPARALLEL: diff --git a/nptl/tst-mutex8.c b/nptl/tst-mutex8.c index 1d288d243c..ef59db55de 100644 --- a/nptl/tst-mutex8.c +++ b/nptl/tst-mutex8.c @@ -127,9 +127,8 @@ check_type (const char *mas, pthread_mutexattr_t *ma) return 1; } - /* Elided mutexes don't fail destroy. If elision is not explicitly disabled - we don't know, so can also not check this. */ -#ifndef ENABLE_LOCK_ELISION + /* Elided mutexes don't fail destroy, but this test is run with + elision disabled so we can test them. */ e = pthread_mutex_destroy (m); if (e == 0) { @@ -142,7 +141,6 @@ check_type (const char *mas, pthread_mutexattr_t *ma) mas); return 1; } -#endif if (pthread_mutex_unlock (m) != 0) { @@ -157,7 +155,6 @@ check_type (const char *mas, pthread_mutexattr_t *ma) } /* Elided mutexes don't fail destroy. */ -#ifndef ENABLE_LOCK_ELISION e = pthread_mutex_destroy (m); if (e == 0) { @@ -171,7 +168,6 @@ mutex_destroy of self-trylocked mutex did not return EBUSY %s\n", mas); return 1; } -#endif if (pthread_mutex_unlock (m) != 0) { @@ -207,7 +203,6 @@ mutex_destroy of self-trylocked mutex did not return EBUSY %s\n", } /* Elided mutexes don't fail destroy. */ -#ifndef ENABLE_LOCK_ELISION e = pthread_mutex_destroy (m); if (e == 0) { @@ -220,7 +215,6 @@ mutex_destroy of self-trylocked mutex did not return EBUSY %s\n", mutex_destroy of condvar-used mutex did not return EBUSY for %s\n", mas); return 1; } -#endif done = true; if (pthread_cond_signal (&c) != 0) @@ -280,7 +274,6 @@ mutex_destroy of condvar-used mutex did not return EBUSY for %s\n", mas); } /* Elided mutexes don't fail destroy. */ -#ifndef ENABLE_LOCK_ELISION e = pthread_mutex_destroy (m); if (e == 0) { @@ -295,7 +288,6 @@ mutex_destroy of condvar-used mutex did not return EBUSY for %s\n", mas); mas); return 1; } -#endif if (pthread_cancel (th) != 0) { diff --git a/sysdeps/powerpc/nptl/elide.h b/sysdeps/powerpc/nptl/elide.h index 1c42814b71..06986ccbce 100644 --- a/sysdeps/powerpc/nptl/elide.h +++ b/sysdeps/powerpc/nptl/elide.h @@ -19,7 +19,6 @@ #ifndef ELIDE_PPC_H # define ELIDE_PPC_H -#ifdef ENABLE_LOCK_ELISION # include # include @@ -114,12 +113,4 @@ __elide_unlock (int is_lock_free) # define ELIDE_UNLOCK(is_lock_free) \ __elide_unlock (is_lock_free) -# else - -# define ELIDE_LOCK(adapt_count, is_lock_free) 0 -# define ELIDE_TRYLOCK(adapt_count, is_lock_free, write) 0 -# define ELIDE_UNLOCK(is_lock_free) 0 - -#endif /* ENABLE_LOCK_ELISION */ - #endif diff --git a/sysdeps/powerpc/powerpc32/sysdep.h b/sysdeps/powerpc/powerpc32/sysdep.h index 965ea43c94..1d2ff732ee 100644 --- a/sysdeps/powerpc/powerpc32/sysdep.h +++ b/sysdeps/powerpc/powerpc32/sysdep.h @@ -90,7 +90,7 @@ GOT_LABEL: ; \ cfi_endproc; \ ASM_SIZE_DIRECTIVE(name) -#if ! IS_IN(rtld) && defined (ENABLE_LOCK_ELISION) +#if ! IS_IN(rtld) # define ABORT_TRANSACTION \ cmpwi 2,0; \ beq 1f; \ diff --git a/sysdeps/powerpc/powerpc64/sysdep.h b/sysdeps/powerpc/powerpc64/sysdep.h index ab5f395cfd..bff184e374 100644 --- a/sysdeps/powerpc/powerpc64/sysdep.h +++ b/sysdeps/powerpc/powerpc64/sysdep.h @@ -263,7 +263,7 @@ LT_LABELSUFFIX(name,_name_end): ; \ TRACEBACK_MASK(name,mask); \ END_2(name) -#if !IS_IN(rtld) && defined (ENABLE_LOCK_ELISION) +#if !IS_IN(rtld) # define ABORT_TRANSACTION \ cmpdi 13,0; \ beq 1f; \ diff --git a/sysdeps/powerpc/sysdep.h b/sysdeps/powerpc/sysdep.h index f07b959eee..d1a9bd9b55 100644 --- a/sysdeps/powerpc/sysdep.h +++ b/sysdeps/powerpc/sysdep.h @@ -21,10 +21,8 @@ */ #define _SYSDEPS_SYSDEP_H 1 #include -#ifdef ENABLE_LOCK_ELISION #include #include -#endif #define PPC_FEATURE_970 (PPC_FEATURE_POWER4 + PPC_FEATURE_HAS_ALTIVEC) @@ -176,7 +174,7 @@ we abort transaction just before syscalls. [1] Documentation/powerpc/transactional_memory.txt [Syscalls] */ -#if !IS_IN(rtld) && defined (ENABLE_LOCK_ELISION) +#if !IS_IN(rtld) # define ABORT_TRANSACTION \ ({ \ if (THREAD_GET_TM_CAPABLE ()) \ diff --git a/sysdeps/s390/configure b/sysdeps/s390/configure index d4a4a3dcf8..74b415f2ab 100644 --- a/sysdeps/s390/configure +++ b/sysdeps/s390/configure @@ -35,7 +35,7 @@ fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_gcc_builtin_tbegin" >&5 $as_echo "$libc_cv_gcc_builtin_tbegin" >&6; } -if test "$enable_lock_elision" = yes && test "$libc_cv_gcc_builtin_tbegin" = no ; then +if test "$libc_cv_gcc_builtin_tbegin" = no ; then critic_missing="$critic_missing The used GCC has no support for __builtin_tbegin, which is needed for lock-elision on target S390." fi diff --git a/sysdeps/s390/configure.ac b/sysdeps/s390/configure.ac index 7d0b5ce46f..1cdb021282 100644 --- a/sysdeps/s390/configure.ac +++ b/sysdeps/s390/configure.ac @@ -26,7 +26,7 @@ else fi rm -f conftest* ]) -if test "$enable_lock_elision" = yes && test "$libc_cv_gcc_builtin_tbegin" = no ; then +if test "$libc_cv_gcc_builtin_tbegin" = no ; then critic_missing="$critic_missing The used GCC has no support for __builtin_tbegin, which is needed for lock-elision on target S390." fi diff --git a/sysdeps/s390/nptl/bits/pthreadtypes-arch.h b/sysdeps/s390/nptl/bits/pthreadtypes-arch.h index 1ae277367d..fd15931974 100644 --- a/sysdeps/s390/nptl/bits/pthreadtypes-arch.h +++ b/sysdeps/s390/nptl/bits/pthreadtypes-arch.h @@ -40,11 +40,7 @@ /* Definitions for internal mutex struct. */ #define __PTHREAD_COMPAT_PADDING_MID #define __PTHREAD_COMPAT_PADDING_END -#ifdef ENABLE_LOCK_ELISION #define __PTHREAD_MUTEX_LOCK_ELISION 1 -#else -#define __PTHREAD_MUTEX_LOCK_ELISION 0 -#endif #define __PTHREAD_MUTEX_NUSERS_AFTER_KIND (__WORDSIZE != 64) #define __PTHREAD_MUTEX_USE_UNION (__WORDSIZE != 64) diff --git a/sysdeps/unix/sysv/linux/powerpc/elision-conf.c b/sysdeps/unix/sysv/linux/powerpc/elision-conf.c index f631f0a035..06361e6b2f 100644 --- a/sysdeps/unix/sysv/linux/powerpc/elision-conf.c +++ b/sysdeps/unix/sysv/linux/powerpc/elision-conf.c @@ -22,6 +22,11 @@ #include #include +#if HAVE_TUNABLES +# define TUNABLE_NAMESPACE elision +#endif +#include + /* Reasonable initial tuning values, may be revised in the future. This is a conservative initial value. */ @@ -50,7 +55,52 @@ struct elision_config __elision_aconf = DEFAULT locks should be automatically use elision in pthread_mutex_lock(). Disabled for suid programs. Only used when elision is available. */ -int __pthread_force_elision attribute_hidden; +int __pthread_force_elision attribute_hidden = 0; + +#if HAVE_TUNABLES +static inline void +__always_inline +do_set_elision_enable (int32_t elision_enable) +{ + /* Enable elision if it's avaliable in hardware. It's not necessary to check + if __libc_enable_secure isn't enabled since elision_enable will be set + according to the default, which is disabled. */ + if (elision_enable == 1) + __pthread_force_elision = (GLRO (dl_hwcap2) + & PPC_FEATURE2_HAS_HTM) ? 1 : 0; +} + +/* The pthread->elision_enable tunable is 0 or 1 indicating that elision + should be disabled or enabled respectively. The feature will only be used + if it's supported by the hardware. */ + +void +TUNABLE_CALLBACK (set_elision_enable) (tunable_val_t *valp) +{ + int32_t elision_enable = (int32_t) valp->numval; + do_set_elision_enable (elision_enable); +} + +#define TUNABLE_CALLBACK_FNDECL(__name, __type) \ +static inline void \ +__always_inline \ +do_set_elision_ ## __name (__type value) \ +{ \ + __elision_aconf.__name = value; \ +} \ +void \ +TUNABLE_CALLBACK (set_elision_ ## __name) (tunable_val_t *valp) \ +{ \ + __type value = (__type) (valp)->numval; \ + do_set_elision_ ## __name (value); \ +} + +TUNABLE_CALLBACK_FNDECL (skip_lock_busy, int32_t); +TUNABLE_CALLBACK_FNDECL (skip_lock_internal_abort, int32_t); +TUNABLE_CALLBACK_FNDECL (skip_lock_out_of_tbegin_retries, int32_t); +TUNABLE_CALLBACK_FNDECL (try_tbegin, int32_t); +TUNABLE_CALLBACK_FNDECL (skip_trylock_internal_abort, int32_t); +#endif /* Initialize elision. */ @@ -59,13 +109,26 @@ elision_init (int argc __attribute__ ((unused)), char **argv __attribute__ ((unused)), char **environ) { -#ifdef ENABLE_LOCK_ELISION - int elision_available = (GLRO (dl_hwcap2) & PPC_FEATURE2_HAS_HTM) ? 1 : 0; - __pthread_force_elision = __libc_enable_secure ? 0 : elision_available; +#if HAVE_TUNABLES + /* Elision depends on tunables and must be explicitly turned on by setting + the appropriate tunable on a supported platform. */ + + TUNABLE_GET (enable, int32_t, + TUNABLE_CALLBACK (set_elision_enable)); + TUNABLE_GET (skip_lock_busy, int32_t, + TUNABLE_CALLBACK (set_elision_skip_lock_busy)); + TUNABLE_GET (skip_lock_internal_abort, int32_t, + TUNABLE_CALLBACK (set_elision_skip_lock_internal_abort)); + TUNABLE_GET (skip_lock_after_retries, int32_t, + TUNABLE_CALLBACK (set_elision_skip_lock_out_of_tbegin_retries)); + TUNABLE_GET (tries, int32_t, + TUNABLE_CALLBACK (set_elision_try_tbegin)); + TUNABLE_GET (skip_trylock_internal_abort, int32_t, + TUNABLE_CALLBACK (set_elision_skip_trylock_internal_abort)); #endif + if (!__pthread_force_elision) - /* Disable elision on rwlocks. */ - __elision_aconf.try_tbegin = 0; + __elision_aconf.try_tbegin = 0; /* Disable elision on rwlocks. */ } #ifdef SHARED diff --git a/sysdeps/unix/sysv/linux/powerpc/force-elision.h b/sysdeps/unix/sysv/linux/powerpc/force-elision.h index 318f7915c7..d1feeeb01e 100644 --- a/sysdeps/unix/sysv/linux/powerpc/force-elision.h +++ b/sysdeps/unix/sysv/linux/powerpc/force-elision.h @@ -16,7 +16,6 @@ License along with the GNU C Library; if not, see . */ -#ifdef ENABLE_LOCK_ELISION /* Automatically enable elision for existing user lock kinds. */ #define FORCE_ELISION(m, s) \ if (__pthread_force_elision \ @@ -25,4 +24,3 @@ mutex->__data.__kind |= PTHREAD_MUTEX_ELISION_NP; \ s; \ } -#endif diff --git a/sysdeps/unix/sysv/linux/s390/Makefile b/sysdeps/unix/sysv/linux/s390/Makefile index c5f544d139..77f38523b5 100644 --- a/sysdeps/unix/sysv/linux/s390/Makefile +++ b/sysdeps/unix/sysv/linux/s390/Makefile @@ -16,7 +16,6 @@ sysdep_routines += dl-vdso endif ifeq ($(subdir),nptl) -ifeq ($(enable-lock-elision),yes) libpthread-sysdep_routines += elision-lock elision-unlock elision-timed \ elision-trylock @@ -26,7 +25,6 @@ CFLAGS-elision-timed.c = $(elision-CFLAGS) CFLAGS-elision-trylock.c = $(elision-CFLAGS) CFLAGS-elision-unlock.c = $(elision-CFLAGS) endif -endif ifeq ($(subdir),misc) tests += tst-ptrace-singleblock diff --git a/sysdeps/unix/sysv/linux/s390/elision-conf.c b/sysdeps/unix/sysv/linux/s390/elision-conf.c index cc0fdef2aa..ab334cb79b 100644 --- a/sysdeps/unix/sysv/linux/s390/elision-conf.c +++ b/sysdeps/unix/sysv/linux/s390/elision-conf.c @@ -22,6 +22,11 @@ #include #include +#if HAVE_TUNABLES +# define TUNABLE_NAMESPACE elision +#endif +#include + /* Reasonable initial tuning values, may be revised in the future. This is a conservative initial value. */ @@ -53,6 +58,50 @@ struct elision_config __elision_aconf = int __pthread_force_elision attribute_hidden = 0; +#if HAVE_TUNABLES +static inline void +__always_inline +do_set_elision_enable (int32_t elision_enable) +{ + /* Enable elision if it's avaliable in hardware. It's not necessary to check + if __libc_enable_secure isn't enabled since elision_enable will be set + according to the default, which is disabled. */ + if (elision_enable == 1) + __pthread_force_elision = (GLRO (dl_hwcap) & HWCAP_S390_TE) ? 1 : 0; +} + +/* The pthread->elision_enable tunable is 0 or 1 indicating that elision + should be disabled or enabled respectively. The feature will only be used + if it's supported by the hardware. */ + +void +TUNABLE_CALLBACK (set_elision_enable) (tunable_val_t *valp) +{ + int32_t elision_enable = (int32_t) valp->numval; + do_set_elision_enable (elision_enable); +} + +#define TUNABLE_CALLBACK_FNDECL(__name, __type) \ +static inline void \ +__always_inline \ +do_set_elision_ ## __name (__type value) \ +{ \ + __elision_aconf.__name = value; \ +} \ +void \ +TUNABLE_CALLBACK (set_elision_ ## __name) (tunable_val_t *valp) \ +{ \ + __type value = (__type) (valp)->numval; \ + do_set_elision_ ## __name (value); \ +} + +TUNABLE_CALLBACK_FNDECL (skip_lock_busy, int32_t); +TUNABLE_CALLBACK_FNDECL (skip_lock_internal_abort, int32_t); +TUNABLE_CALLBACK_FNDECL (skip_lock_out_of_tbegin_retries, int32_t); +TUNABLE_CALLBACK_FNDECL (try_tbegin, int32_t); +TUNABLE_CALLBACK_FNDECL (skip_trylock_internal_abort, int32_t); +#endif + /* Initialize elison. */ static void @@ -60,11 +109,26 @@ elision_init (int argc __attribute__ ((unused)), char **argv __attribute__ ((unused)), char **environ) { - /* Set when the CPU and the kernel supports transactional execution. - When false elision is never attempted. */ - int elision_available = (GLRO (dl_hwcap) & HWCAP_S390_TE) ? 1 : 0; +#if HAVE_TUNABLES + /* Elision depends on tunables and must be explicitly turned on by setting + the appropriate tunable on a supported platform. */ + + TUNABLE_GET (enable, int32_t, + TUNABLE_CALLBACK (set_elision_enable)); + TUNABLE_GET (skip_lock_busy, int32_t, + TUNABLE_CALLBACK (set_elision_skip_lock_busy)); + TUNABLE_GET (skip_lock_internal_abort, int32_t, + TUNABLE_CALLBACK (set_elision_skip_lock_internal_abort)); + TUNABLE_GET (skip_lock_after_retries, int32_t, + TUNABLE_CALLBACK (set_elision_skip_lock_out_of_tbegin_retries)); + TUNABLE_GET (tries, int32_t, + TUNABLE_CALLBACK (set_elision_try_tbegin)); + TUNABLE_GET (skip_trylock_internal_abort, int32_t, + TUNABLE_CALLBACK (set_elision_skip_trylock_internal_abort)); +#endif - __pthread_force_elision = __libc_enable_secure ? 0 : elision_available; + if (!__pthread_force_elision) + __elision_aconf.try_tbegin = 0; /* Disable elision on rwlocks. */ } #ifdef SHARED diff --git a/sysdeps/unix/sysv/linux/s390/elision-conf.h b/sysdeps/unix/sysv/linux/s390/elision-conf.h index 3143f3b114..32f0ed3b8c 100644 --- a/sysdeps/unix/sysv/linux/s390/elision-conf.h +++ b/sysdeps/unix/sysv/linux/s390/elision-conf.h @@ -15,7 +15,6 @@ You should have received a copy of the GNU Lesser General Public License along with the GNU C Library; if not, see . */ -#ifdef ENABLE_LOCK_ELISION #ifndef _ELISION_CONF_H #define _ELISION_CONF_H 1 @@ -41,4 +40,3 @@ extern int __pthread_force_elision attribute_hidden; #define HAVE_ELISION 1 #endif -#endif diff --git a/sysdeps/unix/sysv/linux/s390/force-elision.h b/sysdeps/unix/sysv/linux/s390/force-elision.h index 3ae3bcd566..8e1e33e1c5 100644 --- a/sysdeps/unix/sysv/linux/s390/force-elision.h +++ b/sysdeps/unix/sysv/linux/s390/force-elision.h @@ -16,7 +16,6 @@ License along with the GNU C Library; if not, see . */ -#ifdef ENABLE_LOCK_ELISION /* Automatically enable elision for existing user lock kinds. */ #define FORCE_ELISION(m, s) \ if (__pthread_force_elision \ @@ -25,4 +24,3 @@ mutex->__data.__kind |= PTHREAD_MUTEX_ELISION_NP; \ s; \ } -#endif diff --git a/sysdeps/unix/sysv/linux/s390/lowlevellock.h b/sysdeps/unix/sysv/linux/s390/lowlevellock.h index 604137f7f2..48f87a85f5 100644 --- a/sysdeps/unix/sysv/linux/s390/lowlevellock.h +++ b/sysdeps/unix/sysv/linux/s390/lowlevellock.h @@ -22,7 +22,6 @@ #include /* Transactional lock elision definitions. */ -# ifdef ENABLE_LOCK_ELISION extern int __lll_timedlock_elision (int *futex, short *adapt_count, const struct timespec *timeout, int private) attribute_hidden; @@ -45,6 +44,5 @@ extern int __lll_trylock_elision(int *futex, short *adapt_count) __lll_unlock_elision (&(futex), &(adapt_count), private) # define lll_trylock_elision(futex, adapt_count) \ __lll_trylock_elision(&(futex), &(adapt_count)) -# endif /* ENABLE_LOCK_ELISION */ #endif /* lowlevellock.h */ diff --git a/sysdeps/unix/sysv/linux/x86/elision-conf.c b/sysdeps/unix/sysv/linux/x86/elision-conf.c index 673b0005a7..7e9fbf9382 100644 --- a/sysdeps/unix/sysv/linux/x86/elision-conf.c +++ b/sysdeps/unix/sysv/linux/x86/elision-conf.c @@ -22,6 +22,11 @@ #include #include +#if HAVE_TUNABLES +# define TUNABLE_NAMESPACE elision +#endif +#include + /* Reasonable initial tuning values, may be revised in the future. This is a conservative initial value. */ @@ -48,21 +53,76 @@ struct elision_config __elision_aconf = pthread_mutex_lock(). Disabled for suid programs. Only used when elision is available. */ -int __pthread_force_elision attribute_hidden; +int __pthread_force_elision attribute_hidden = 0; + +#if HAVE_TUNABLES +static inline void +__always_inline +do_set_elision_enable (int32_t elision_enable) +{ + /* Enable elision if it's avaliable in hardware. It's not necessary to check + if __libc_enable_secure isn't enabled since elision_enable will be set + according to the default, which is disabled. */ + if (elision_enable == 1) + __pthread_force_elision = HAS_CPU_FEATURE (RTM) ? 1 : 0; +} + +/* The pthread->elision_enable tunable is 0 or 1 indicating that elision + should be disabled or enabled respectively. The feature will only be used + if it's supported by the hardware. */ -/* Initialize elison. */ +void +TUNABLE_CALLBACK (set_elision_enable) (tunable_val_t *valp) +{ + int32_t elision_enable = (int32_t) valp->numval; + do_set_elision_enable (elision_enable); +} + +#define TUNABLE_CALLBACK_FNDECL(__name, __type) \ +static inline void \ +__always_inline \ +do_set_elision_ ## __name (__type value) \ +{ \ + __elision_aconf.__name = value; \ +} \ +void \ +TUNABLE_CALLBACK (set_elision_ ## __name) (tunable_val_t *valp) \ +{ \ + __type value = (__type) (valp)->numval; \ + do_set_elision_ ## __name (value); \ +} + +TUNABLE_CALLBACK_FNDECL (skip_lock_busy, int32_t); +TUNABLE_CALLBACK_FNDECL (skip_lock_internal_abort, int32_t); +TUNABLE_CALLBACK_FNDECL (retry_try_xbegin, int32_t); +TUNABLE_CALLBACK_FNDECL (skip_trylock_internal_abort, int32_t); +#endif + +/* Initialize elision. */ static void elision_init (int argc __attribute__ ((unused)), char **argv __attribute__ ((unused)), char **environ) { - int elision_available = HAS_CPU_FEATURE (RTM); -#ifdef ENABLE_LOCK_ELISION - __pthread_force_elision = __libc_enable_secure ? 0 : elision_available; +#if HAVE_TUNABLES + /* Elision depends on tunables and must be explicitly turned on by setting + the appropriate tunable on a supported platform. */ + + TUNABLE_GET (enable, int32_t, + TUNABLE_CALLBACK (set_elision_enable)); + TUNABLE_GET (skip_lock_busy, int32_t, + TUNABLE_CALLBACK (set_elision_skip_lock_busy)); + TUNABLE_GET (skip_lock_internal_abort, int32_t, + TUNABLE_CALLBACK (set_elision_skip_lock_internal_abort)); + TUNABLE_GET (tries, int32_t, + TUNABLE_CALLBACK (set_elision_retry_try_xbegin)); + TUNABLE_GET (skip_trylock_internal_abort, int32_t, + TUNABLE_CALLBACK (set_elision_skip_trylock_internal_abort)); #endif - if (!elision_available) - __elision_aconf.retry_try_xbegin = 0; /* Disable elision on rwlocks */ + + if (!__pthread_force_elision) + __elision_aconf.retry_try_xbegin = 0; /* Disable elision on rwlocks. */ } #ifdef SHARED -- cgit v1.2.3 From 8df5d34720dd71e934545bade879e04697830757 Mon Sep 17 00:00:00 2001 From: Joseph Myers Date: Tue, 12 Dec 2017 13:56:47 +0000 Subject: Remove --with-fp / --without-fp. There is a configure option --without-fp that specifies that nofpu sysdeps directories should be used instead of fpu directories. For most glibc configurations, this option is of no use: either there is no valid nofpu variant of that configuration, or there are no fpu or nofpu sysdeps directories for that processor and so the option does nothing. For a few configurations, if you are using a soft-float compiler this option is required, and failing to use it generally results in compilation errors from inline asm using unavailable floating-point instructions. We're moving away from --with-cpu to configuring glibc based on how the compiler generates code, and it is natural to do so for --without-fp as well; in most cases the soft-float and hard-float ABIs are incompatible so you have no hope of building a working glibc with an inappropriately configured compiler or libgcc. This patch eliminates --without-fp, replacing it entirely by automatic configuration based on the compiler. Configurations for which this is relevant (coldfire / mips / powerpc32 / sh) define a variable with_fp_cond in their preconfigure fragments (under the same conditions under which those fragments do anything); this is a preprocessor conditional which the toplevel configure script then uses in a test to determine which sysdeps directories to use. The config.make with-fp variable remains. It's used only by powerpc (sysdeps/powerpc/powerpc32/Makefile) to add -mhard-float to various flags variables. For powerpc, -mcpu= options can imply use of soft-float. That could be an issue if you want to build for e.g. 476fp, but are using --with-cpu=476 because there isn't a 476fp sysdeps directory. If in future we eliminate --with-cpu and replace it entirely by testing the compiler, it would be natural at that point to eliminate that code as well (as the user should then just use a compiler defaulting to 476fp and the 476 sysdeps directory would be used automatically). Tested for x86_64, and tested with build-many-glibcs.py that installed shared libraries are unchanged by this patch. * configure.ac (--with-fp): Remove configure option. (with_fp_cond): New variable. (libc_cv_with_fp): New configure test. Use this variable instead of with_fp. * configure: Regenerated. * config.make.in (with-fp): Use @libc_cv_with_fp@. * manual/install.texi (Configuring and compiling): Remove --without-fp. * INSTALL: Regenerated. * sysdeps/m68k/preconfigure (with_fp_cond): Define for ColdFire. * sysdeps/mips/preconfigure (with_fp_cond): Define. * sysdeps/powerpc/preconfigure (with_fp_cond): Define for 32-bit. * sysdeps/sh/preconfigure (with_fp_cond): Define. * scripts/build-many-glibcs.py (Context.add_all_configs): Do not use --without-fp to configure glibc. --- ChangeLog | 18 ++++++++++++++++ INSTALL | 4 ---- NEWS | 5 +++++ config.make.in | 2 +- configure | 44 +++++++++++++++++++++++++++----------- configure.ac | 31 +++++++++++++++++++++------ manual/install.texi | 4 ---- scripts/build-many-glibcs.py | 51 +++++++++++++++----------------------------- sysdeps/m68k/preconfigure | 6 +++++- sysdeps/mips/preconfigure | 2 ++ sysdeps/powerpc/preconfigure | 1 + sysdeps/sh/preconfigure | 10 +++++++-- 12 files changed, 112 insertions(+), 66 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 90e47f7877..7a54848d30 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,21 @@ +2017-12-12 Joseph Myers + + * configure.ac (--with-fp): Remove configure option. + (with_fp_cond): New variable. + (libc_cv_with_fp): New configure test. Use this variable instead + of with_fp. + * configure: Regenerated. + * config.make.in (with-fp): Use @libc_cv_with_fp@. + * manual/install.texi (Configuring and compiling): Remove + --without-fp. + * INSTALL: Regenerated. + * sysdeps/m68k/preconfigure (with_fp_cond): Define for ColdFire. + * sysdeps/mips/preconfigure (with_fp_cond): Define. + * sysdeps/powerpc/preconfigure (with_fp_cond): Define for 32-bit. + * sysdeps/sh/preconfigure (with_fp_cond): Define. + * scripts/build-many-glibcs.py (Context.add_all_configs): Do not + use --without-fp to configure glibc. + 2017-12-12 Rical Jasan * manual/locale.texi (nl_langinfo): Fix a typo. diff --git a/INSTALL b/INSTALL index e59c11dabd..80306de35c 100644 --- a/INSTALL +++ b/INSTALL @@ -81,10 +81,6 @@ will be used, and CFLAGS sets optimization options for the compiler. library will still be usable, but functionality may be lost--for example, you can't build a shared libc with old binutils. -'--without-fp' - Use this option if your computer lacks hardware floating-point - support and your operating system does not emulate an FPU. - '--disable-shared' Don't build shared libraries even if it is possible. Not all systems support shared libraries; you need ELF support and diff --git a/NEWS b/NEWS index 25a2116cdd..c6d859ca75 100644 --- a/NEWS +++ b/NEWS @@ -76,6 +76,11 @@ Deprecated and removed features, and other changes affecting compatibility: glibc has been removed. The --enable-add-ons configure option is now ignored. +* The --without-fp configure option is now ignored. Whether hardware + floating-point instructions are used is now configured based on whether + the compiler used at configure time (without any options implied by a + --with-cpu= configure option) uses such instructions. + * The res_hnok, res_dnok, res_mailok and res_ownok functions now check that the specified string can be parsed as a domain name. diff --git a/config.make.in b/config.make.in index 9da77d1efa..04513dbb74 100644 --- a/config.make.in +++ b/config.make.in @@ -57,7 +57,7 @@ have-z-execstack = @libc_cv_z_execstack@ have-protected-data = @libc_cv_protected_data@ have-insert = @libc_cv_insert@ have-glob-dat-reloc = @libc_cv_has_glob_dat@ -with-fp = @with_fp@ +with-fp = @libc_cv_with_fp@ enable-timezone-tools = @enable_timezone_tools@ unwind-find-fde = @libc_cv_gcc_unwind_find_fde@ have-fpie = @libc_cv_fpie@ diff --git a/configure b/configure index dd8b8c95e6..4a85706862 100755 --- a/configure +++ b/configure @@ -666,6 +666,7 @@ multi_arch no_stack_protector stack_protector libc_cv_ssp +libc_cv_with_fp base_machine have_tunables build_pt_chown @@ -683,7 +684,6 @@ hardcoded_path_in_tests enable_timezone_tools use_default_link sysheaders -with_fp ac_ct_CXX CXXFLAGS CXX @@ -756,7 +756,6 @@ with_bugurl with_gd with_gd_include with_gd_lib -with_fp with_binutils with_selinux with_headers @@ -1467,7 +1466,6 @@ Optional Packages: --with-gd=DIR find libgd include dir and library with prefix DIR --with-gd-include=DIR find libgd include files in DIR --with-gd-lib=DIR find libgd library files in DIR - --with-fp if using floating-point hardware [default=yes] --with-binutils=PATH specify location of binutils (as and ld) --with-selinux if building with SELinux support --with-headers=PATH location of system headers to use (for example @@ -3294,15 +3292,6 @@ libgd-LDFLAGS = $libgd_ldflags" fi -# Check whether --with-fp was given. -if test "${with_fp+set}" = set; then : - withval=$with_fp; with_fp=$withval -else - with_fp=yes -fi - - - # Check whether --with-binutils was given. if test "${with_binutils+set}" = set; then : withval=$with_binutils; path_binutils=$withval @@ -3778,6 +3767,11 @@ fi # check below. libc_config_ok=no +# A preconfigure script for a system that may or may not use fpu +# sysdeps directories sets this to a preprocessor conditional for +# whether to use such directories. +with_fp_cond=1 + if frags=`ls -d $srcdir/sysdeps/*/preconfigure 2> /dev/null` then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for sysdeps preconfigure fragments" >&5 @@ -3816,6 +3810,30 @@ fi test -n "$base_machine" || base_machine=$machine +# Determine whether to use fpu or nofpu sysdeps directories. +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for use of fpu sysdeps directories" >&5 +$as_echo_n "checking for use of fpu sysdeps directories... " >&6; } +if ${libc_cv_with_fp+:} false; then : + $as_echo_n "(cached) " >&6 +else + cat > conftest.c <&5 2>&5 ; then + libc_cv_with_fp=yes +fi +rm -f conftest* +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_with_fp" >&5 +$as_echo "$libc_cv_with_fp" >&6; } + + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -fstack-protector" >&5 $as_echo_n "checking for -fstack-protector... " >&6; } if ${libc_cv_ssp+:} false; then : @@ -4085,7 +4103,7 @@ tail=$machine${submachine:+/$submachine} while m=`echo $tail | sed 's@^\(.*\)/\([^/]*\)$@& \1@'`; test -n "$m"; do set $m # Prepend the machine's FPU directory unless --without-fp. - if test "$with_fp" = yes; then + if test "$libc_cv_with_fp" = yes; then maybe_fpu=/fpu else maybe_fpu=/nofpu diff --git a/configure.ac b/configure.ac index f85a50da53..edf662715b 100644 --- a/configure.ac +++ b/configure.ac @@ -127,12 +127,6 @@ libgd-LDFLAGS = $libgd_ldflags" fi dnl Arguments to specify presence of other packages/features. -AC_ARG_WITH([fp], - AC_HELP_STRING([--with-fp], - [if using floating-point hardware @<:@default=yes@:>@]), - [with_fp=$withval], - [with_fp=yes]) -AC_SUBST(with_fp) AC_ARG_WITH([binutils], AC_HELP_STRING([--with-binutils=PATH], [specify location of binutils (as and ld)]), @@ -489,6 +483,11 @@ AC_ARG_WITH([cpu], # check below. libc_config_ok=no +# A preconfigure script for a system that may or may not use fpu +# sysdeps directories sets this to a preprocessor conditional for +# whether to use such directories. +with_fp_cond=1 + dnl Let sysdeps/*/preconfigure act here. LIBC_PRECONFIGURE([$srcdir], [for sysdeps]) @@ -517,6 +516,24 @@ fi test -n "$base_machine" || base_machine=$machine AC_SUBST(base_machine) +# Determine whether to use fpu or nofpu sysdeps directories. +AC_CACHE_CHECK([for use of fpu sysdeps directories], + libc_cv_with_fp, [dnl +cat > conftest.c <&AS_MESSAGE_LOG_FD 2>&AS_MESSAGE_LOG_FD ; then + libc_cv_with_fp=yes +fi +rm -f conftest*]) +AC_SUBST(libc_cv_with_fp) + AC_CACHE_CHECK(for -fstack-protector, libc_cv_ssp, [dnl LIBC_TRY_CC_OPTION([$CFLAGS $CPPFLAGS -Werror -fstack-protector], [libc_cv_ssp=yes], @@ -720,7 +737,7 @@ tail=$machine${submachine:+/$submachine} while m=`echo $tail | sed 's@^\(.*\)/\([^/]*\)$@& \1@'`; test -n "$m"; do set $m # Prepend the machine's FPU directory unless --without-fp. - if test "$with_fp" = yes; then + if test "$libc_cv_with_fp" = yes; then maybe_fpu=/fpu else maybe_fpu=/nofpu diff --git a/manual/install.texi b/manual/install.texi index a3cb09dafc..e81f1c50c3 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -106,10 +106,6 @@ problem and suppress these constructs, so that the library will still be usable, but functionality may be lost---for example, you can't build a shared libc with old binutils. -@item --without-fp -Use this option if your computer lacks hardware floating-point support -and your operating system does not emulate an FPU. - @c disable static doesn't work currently @c @item --disable-static @c Don't build static libraries. Static libraries aren't that useful these diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py index 59972a0602..599e12e355 100755 --- a/scripts/build-many-glibcs.py +++ b/scripts/build-many-glibcs.py @@ -216,15 +216,12 @@ class Context(object): os_name='linux-gnu', variant='soft', gcc_cfg=['--with-mips-plt', '--with-float=soft'], - glibcs=[{'variant': 'n32-soft', - 'cfg': ['--without-fp']}, + glibcs=[{'variant': 'n32-soft'}, {'variant': 'soft', 'arch': 'mips', - 'ccopts': '-mabi=32', - 'cfg': ['--without-fp']}, + 'ccopts': '-mabi=32'}, {'variant': 'n64-soft', - 'ccopts': '-mabi=64', - 'cfg': ['--without-fp']}]) + 'ccopts': '-mabi=64'}]) self.add_config(arch='mips64', os_name='linux-gnu', variant='nan2008', @@ -244,15 +241,12 @@ class Context(object): '--with-arch-64=mips64r2', '--with-arch-32=mips32r2', '--with-float=soft'], - glibcs=[{'variant': 'n32-nan2008-soft', - 'cfg': ['--without-fp']}, + glibcs=[{'variant': 'n32-nan2008-soft'}, {'variant': 'nan2008-soft', 'arch': 'mips', - 'ccopts': '-mabi=32', - 'cfg': ['--without-fp']}, + 'ccopts': '-mabi=32'}, {'variant': 'n64-nan2008-soft', - 'ccopts': '-mabi=64', - 'cfg': ['--without-fp']}]) + 'ccopts': '-mabi=64'}]) self.add_config(arch='mips64el', os_name='linux-gnu', gcc_cfg=['--with-mips-plt'], @@ -265,15 +259,12 @@ class Context(object): os_name='linux-gnu', variant='soft', gcc_cfg=['--with-mips-plt', '--with-float=soft'], - glibcs=[{'variant': 'n32-soft', - 'cfg': ['--without-fp']}, + glibcs=[{'variant': 'n32-soft'}, {'variant': 'soft', 'arch': 'mipsel', - 'ccopts': '-mabi=32', - 'cfg': ['--without-fp']}, + 'ccopts': '-mabi=32'}, {'variant': 'n64-soft', - 'ccopts': '-mabi=64', - 'cfg': ['--without-fp']}]) + 'ccopts': '-mabi=64'}]) self.add_config(arch='mips64el', os_name='linux-gnu', variant='nan2008', @@ -293,15 +284,12 @@ class Context(object): '--with-arch-64=mips64r2', '--with-arch-32=mips32r2', '--with-float=soft'], - glibcs=[{'variant': 'n32-nan2008-soft', - 'cfg': ['--without-fp']}, + glibcs=[{'variant': 'n32-nan2008-soft'}, {'variant': 'nan2008-soft', 'arch': 'mipsel', - 'ccopts': '-mabi=32', - 'cfg': ['--without-fp']}, + 'ccopts': '-mabi=32'}, {'variant': 'n64-nan2008-soft', - 'ccopts': '-mabi=64', - 'cfg': ['--without-fp']}]) + 'ccopts': '-mabi=64'}]) self.add_config(arch='nios2', os_name='linux-gnu') self.add_config(arch='powerpc', @@ -314,8 +302,7 @@ class Context(object): os_name='linux-gnu', variant='soft', gcc_cfg=['--disable-multilib', '--with-float=soft', - '--enable-secureplt'], - glibcs=[{'variant': 'soft', 'cfg': ['--without-fp']}]) + '--enable-secureplt']) self.add_config(arch='powerpc64', os_name='linux-gnu', gcc_cfg=['--disable-multilib', '--enable-secureplt']) @@ -325,13 +312,11 @@ class Context(object): self.add_config(arch='powerpc', os_name='linux-gnuspe', gcc_cfg=['--disable-multilib', '--enable-secureplt', - '--enable-e500-double'], - glibcs=[{'cfg': ['--without-fp']}]) + '--enable-e500-double']) self.add_config(arch='powerpc', os_name='linux-gnuspe', variant='e500v1', - gcc_cfg=['--disable-multilib', '--enable-secureplt'], - glibcs=[{'variant': 'e500v1', 'cfg': ['--without-fp']}]) + gcc_cfg=['--disable-multilib', '--enable-secureplt']) self.add_config(arch='s390x', os_name='linux-gnu', glibcs=[{}, @@ -347,13 +332,11 @@ class Context(object): self.add_config(arch='sh4', os_name='linux-gnu', variant='soft', - gcc_cfg=['--without-fp'], - glibcs=[{'variant': 'soft', 'cfg': ['--without-fp']}]) + gcc_cfg=['--without-fp']) self.add_config(arch='sh4eb', os_name='linux-gnu', variant='soft', - gcc_cfg=['--without-fp'], - glibcs=[{'variant': 'soft', 'cfg': ['--without-fp']}]) + gcc_cfg=['--without-fp']) self.add_config(arch='sparc64', os_name='linux-gnu', glibcs=[{}, diff --git a/sysdeps/m68k/preconfigure b/sysdeps/m68k/preconfigure index 94fc1aabc2..1028dac080 100644 --- a/sysdeps/m68k/preconfigure +++ b/sysdeps/m68k/preconfigure @@ -13,5 +13,9 @@ m68k) variant=`(echo "#ifdef __mcoldfire__" echo >&2 "Cannot determine m68k processor variant" exit 1 fi - base_machine=m68k machine=m68k/$variant ;; + base_machine=m68k machine=m68k/$variant + if test "$variant" = "coldfire"; then + with_fp_cond="defined __mcffpu__" + fi + ;; esac diff --git a/sysdeps/mips/preconfigure b/sysdeps/mips/preconfigure index c118592b27..48fc32ed5b 100644 --- a/sysdeps/mips/preconfigure +++ b/sysdeps/mips/preconfigure @@ -25,5 +25,7 @@ mips*) if test "$abiflag" != "_ABIO32" -a "$mips16flag" = "1"; then as_fn_error $? "MIPS16 is only supported with the o32 ABI." "$LINENO" 5 fi + + with_fp_cond="defined __mips_hard_float" ;; esac diff --git a/sysdeps/powerpc/preconfigure b/sysdeps/powerpc/preconfigure index 7de2eafd52..0030bfd0c0 100644 --- a/sysdeps/powerpc/preconfigure +++ b/sysdeps/powerpc/preconfigure @@ -16,5 +16,6 @@ powerpc*) base_machine=powerpc machine=powerpc/powerpc32 fi rm -f conftest.i + with_fp_cond="!defined __NO_FPRS__" ;; esac diff --git a/sysdeps/sh/preconfigure b/sysdeps/sh/preconfigure index c1f6537797..16e29273df 100644 --- a/sysdeps/sh/preconfigure +++ b/sysdeps/sh/preconfigure @@ -1,6 +1,12 @@ # preconfigure fragment for sh. case "$machine" in -sh3*) base_machine=sh machine=sh/sh3 ;; -sh4*) base_machine=sh machine=sh/sh4 ;; +sh3*) base_machine=sh + machine=sh/sh3 + with_fp_cond="defined __SH_FPU_ANY__" + ;; +sh4*) base_machine=sh + machine=sh/sh4 + with_fp_cond="defined __SH_FPU_ANY__" + ;; esac -- cgit v1.2.3 From 9d7a3741c9e59eba87fb3ca6b9f979befce07826 Mon Sep 17 00:00:00 2001 From: "H.J. Lu" Date: Fri, 15 Dec 2017 16:59:33 -0800 Subject: Add --enable-static-pie configure option to build static PIE [BZ #19574] Static PIE extends address space layout randomization to static executables. It provides additional security hardening benefits at the cost of some memory and performance. Dynamic linker, ld.so, is a standalone program which can be loaded at any address. This patch adds a configure option, --enable-static-pie, to embed the part of ld.so in static executable to create static position independent executable (static PIE). A static PIE is similar to static executable, but can be loaded at any address without help from a dynamic linker. When --enable-static-pie is used to configure glibc, libc.a is built as PIE and all static executables, including tests, are built as static PIE. The resulting libc.a can be used together with GCC 8 or above to build static PIE with the compiler option, -static-pie. But GCC 8 isn't required to build glibc with --enable-static-pie. Only GCC with PIE support is needed. When an older GCC is used to build glibc with --enable-static-pie, proper input files are passed to linker to create static executables as static PIE, together with "-z text" to prevent dynamic relocations in read-only segments, which are not allowed in static PIE. The following changes are made for static PIE: 1. Add a new function, _dl_relocate_static_pie, to: a. Get the run-time load address. b. Read the dynamic section. c. Perform dynamic relocations. Dynamic linker also performs these steps. But static PIE doesn't load any shared objects. 2. Call _dl_relocate_static_pie at entrance of LIBC_START_MAIN in libc.a. crt1.o, which is used to create dynamic and non-PIE static executables, is updated to include a dummy _dl_relocate_static_pie. rcrt1.o is added to create static PIE, which will link in the real _dl_relocate_static_pie. grcrt1.o is also added to create static PIE with -pg. GCC 8 has been updated to support rcrt1.o and grcrt1.o for static PIE. Static PIE can work on all architectures which support PIE, provided: 1. Target must support accessing of local functions without dynamic relocations, which is needed in start.S to call __libc_start_main with function addresses of __libc_csu_init, __libc_csu_fini and main. All functions in static PIE are local functions. If PIE start.S can't reach main () defined in a shared object, the code sequence: pass address of local_main to __libc_start_main ... local_main: tail call to main via PLT can be used. 2. start.S is updated to check PIC instead SHARED for PIC code path and avoid dynamic relocation, when PIC is defined and SHARED isn't defined, to support static PIE. 3. All assembly codes are updated check PIC instead SHARED for PIC code path to avoid dynamic relocations in read-only sections. 4. All assembly codes are updated check SHARED instead PIC for static symbol name. 5. elf_machine_load_address in dl-machine.h are updated to support static PIE. 6. __brk works without TLS nor dynamic relocations in read-only section so that it can be used by __libc_setup_tls to initializes TLS in static PIE. NB: When glibc is built with GCC defaulted to PIE, libc.a is compiled with -fPIE, regardless if --enable-static-pie is used to configure glibc. When glibc is configured with --enable-static-pie, libc.a is compiled with -fPIE, regardless whether GCC defaults to PIE or not. The same libc.a can be used to build both static executable and static PIE. There is no need for separate PIE copy of libc.a. On x86-64, the normal static sln: text data bss dec hex filename 625425 8284 5456 639165 9c0bd elf/sln the static PIE sln: text data bss dec hex filename 657626 20636 5392 683654 a6e86 elf/sln The code size is increased by 5% and the binary size is increased by 7%. Linker requirements to build glibc with --enable-static-pie: 1. Linker supports --no-dynamic-linker to remove PT_INTERP segment from static PIE. 2. Linker can create working static PIE. The x86-64 linker needs the fix for https://sourceware.org/bugzilla/show_bug.cgi?id=21782 The i386 linker needs to be able to convert "movl main@GOT(%ebx), %eax" to "leal main@GOTOFF(%ebx), %eax" if main is defined locally. Binutils 2.29 or above are OK for i686 and x86-64. But linker status for other targets need to be verified. 3. Linker should resolve undefined weak symbols to 0 in static PIE: https://sourceware.org/bugzilla/show_bug.cgi?id=22269 4. Many ELF backend linkers incorrectly check bfd_link_pic for TLS relocations, which should check bfd_link_executable instead: https://sourceware.org/bugzilla/show_bug.cgi?id=22263 Tested on aarch64, i686 and x86-64. Using GCC 7 and binutils master branch, build-many-glibcs.py with --enable-static-pie with all patches for static PIE applied have the following build successes: PASS: glibcs-aarch64_be-linux-gnu build PASS: glibcs-aarch64-linux-gnu build PASS: glibcs-armeb-linux-gnueabi-be8 build PASS: glibcs-armeb-linux-gnueabi build PASS: glibcs-armeb-linux-gnueabihf-be8 build PASS: glibcs-armeb-linux-gnueabihf build PASS: glibcs-arm-linux-gnueabi build PASS: glibcs-arm-linux-gnueabihf build PASS: glibcs-arm-linux-gnueabihf-v7a build PASS: glibcs-arm-linux-gnueabihf-v7a-disable-multi-arch build PASS: glibcs-m68k-linux-gnu build PASS: glibcs-microblazeel-linux-gnu build PASS: glibcs-microblaze-linux-gnu build PASS: glibcs-mips64el-linux-gnu-n32 build PASS: glibcs-mips64el-linux-gnu-n32-nan2008 build PASS: glibcs-mips64el-linux-gnu-n32-nan2008-soft build PASS: glibcs-mips64el-linux-gnu-n32-soft build PASS: glibcs-mips64el-linux-gnu-n64 build PASS: glibcs-mips64el-linux-gnu-n64-nan2008 build PASS: glibcs-mips64el-linux-gnu-n64-nan2008-soft build PASS: glibcs-mips64el-linux-gnu-n64-soft build PASS: glibcs-mips64-linux-gnu-n32 build PASS: glibcs-mips64-linux-gnu-n32-nan2008 build PASS: glibcs-mips64-linux-gnu-n32-nan2008-soft build PASS: glibcs-mips64-linux-gnu-n32-soft build PASS: glibcs-mips64-linux-gnu-n64 build PASS: glibcs-mips64-linux-gnu-n64-nan2008 build PASS: glibcs-mips64-linux-gnu-n64-nan2008-soft build PASS: glibcs-mips64-linux-gnu-n64-soft build PASS: glibcs-mipsel-linux-gnu build PASS: glibcs-mipsel-linux-gnu-nan2008 build PASS: glibcs-mipsel-linux-gnu-nan2008-soft build PASS: glibcs-mipsel-linux-gnu-soft build PASS: glibcs-mips-linux-gnu build PASS: glibcs-mips-linux-gnu-nan2008 build PASS: glibcs-mips-linux-gnu-nan2008-soft build PASS: glibcs-mips-linux-gnu-soft build PASS: glibcs-nios2-linux-gnu build PASS: glibcs-powerpc64le-linux-gnu build PASS: glibcs-powerpc64-linux-gnu build PASS: glibcs-tilegxbe-linux-gnu-32 build PASS: glibcs-tilegxbe-linux-gnu build PASS: glibcs-tilegx-linux-gnu-32 build PASS: glibcs-tilegx-linux-gnu build PASS: glibcs-tilepro-linux-gnu build and the following build failures: FAIL: glibcs-alpha-linux-gnu build elf/sln is failed to link due to: assertion fail bfd/elf64-alpha.c:4125 This is caused by linker bug and/or non-PIC code in PIE libc.a. FAIL: glibcs-hppa-linux-gnu build elf/sln is failed to link due to: collect2: fatal error: ld terminated with signal 11 [Segmentation fault] https://sourceware.org/bugzilla/show_bug.cgi?id=22537 FAIL: glibcs-ia64-linux-gnu build elf/sln is failed to link due to: collect2: fatal error: ld terminated with signal 11 [Segmentation fault] FAIL: glibcs-powerpc-linux-gnu build FAIL: glibcs-powerpc-linux-gnu-soft build FAIL: glibcs-powerpc-linux-gnuspe build FAIL: glibcs-powerpc-linux-gnuspe-e500v1 build elf/sln is failed to link due to: ld: read-only segment has dynamic relocations. This is caused by linker bug and/or non-PIC code in PIE libc.a. See: https://sourceware.org/bugzilla/show_bug.cgi?id=22264 FAIL: glibcs-powerpc-linux-gnu-power4 build elf/sln is failed to link due to: findlocale.c:96:(.text+0x22c): @local call to ifunc memchr This is caused by linker bug and/or non-PIC code in PIE libc.a. FAIL: glibcs-s390-linux-gnu build elf/sln is failed to link due to: collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped assertion fail bfd/elflink.c:14299 This is caused by linker bug and/or non-PIC code in PIE libc.a. FAIL: glibcs-sh3eb-linux-gnu build FAIL: glibcs-sh3-linux-gnu build FAIL: glibcs-sh4eb-linux-gnu build FAIL: glibcs-sh4eb-linux-gnu-soft build FAIL: glibcs-sh4-linux-gnu build FAIL: glibcs-sh4-linux-gnu-soft build elf/sln is failed to link due to: ld: read-only segment has dynamic relocations. This is caused by linker bug and/or non-PIC code in PIE libc.a. See: https://sourceware.org/bugzilla/show_bug.cgi?id=22263 Also TLS code sequence in SH assembly syscalls in glibc doesn't match TLS code sequence expected by ld: https://sourceware.org/bugzilla/show_bug.cgi?id=22270 FAIL: glibcs-sparc64-linux-gnu build FAIL: glibcs-sparcv9-linux-gnu build FAIL: glibcs-tilegxbe-linux-gnu build FAIL: glibcs-tilegxbe-linux-gnu-32 build FAIL: glibcs-tilegx-linux-gnu build FAIL: glibcs-tilegx-linux-gnu-32 build FAIL: glibcs-tilepro-linux-gnu build elf/sln is failed to link due to: ld: read-only segment has dynamic relocations. This is caused by linker bug and/or non-PIC code in PIE libc.a. See: https://sourceware.org/bugzilla/show_bug.cgi?id=22263 [BZ #19574] * INSTALL: Regenerated. * Makeconfig (real-static-start-installed-name): New. (pic-default): Updated for --enable-static-pie. (pie-default): New for --enable-static-pie. (default-pie-ldflag): Likewise. (+link-static-before-libc): Replace $(DEFAULT-LDFLAGS-$(@F)) with $(if $($(@F)-no-pie),$(no-pie-ldflag),$(default-pie-ldflag)). Replace $(static-start-installed-name) with $(real-static-start-installed-name). (+prectorT): Updated for --enable-static-pie. (+postctorT): Likewise. (CFLAGS-.o): Add $(pie-default). (CFLAGS-.op): Likewise. * NEWS: Mention --enable-static-pie. * config.h.in (ENABLE_STATIC_PIE): New. * configure.ac (--enable-static-pie): New configure option. (have-no-dynamic-linker): New LIBC_CONFIG_VAR. (have-static-pie): Likewise. Enable static PIE if linker supports --no-dynamic-linker. (ENABLE_STATIC_PIE): New AC_DEFINE. (enable-static-pie): New LIBC_CONFIG_VAR. * configure: Regenerated. * csu/Makefile (omit-deps): Add r$(start-installed-name) and gr$(start-installed-name) for --enable-static-pie. (extra-objs): Likewise. (install-lib): Likewise. (extra-objs): Add static-reloc.o and static-reloc.os ($(objpfx)$(start-installed-name)): Also depend on $(objpfx)static-reloc.o. ($(objpfx)r$(start-installed-name)): New. ($(objpfx)g$(start-installed-name)): Also depend on $(objpfx)static-reloc.os. ($(objpfx)gr$(start-installed-name)): New. * csu/libc-start.c (LIBC_START_MAIN): Call _dl_relocate_static_pie in libc.a. * csu/libc-tls.c (__libc_setup_tls): Add main_map->l_addr to initimage. * csu/static-reloc.c: New file. * elf/Makefile (routines): Add dl-reloc-static-pie. (elide-routines.os): Likewise. (DEFAULT-LDFLAGS-tst-tls1-static-non-pie): Removed. (tst-tls1-static-non-pie-no-pie): New. * elf/dl-reloc-static-pie.c: New file. * elf/dl-support.c (_dl_get_dl_main_map): New function. * elf/dynamic-link.h (ELF_DURING_STARTUP): Also check STATIC_PIE_BOOTSTRAP. * elf/get-dynamic-info.h (elf_get_dynamic_info): Likewise. * gmon/Makefile (tests): Add tst-gmon-static-pie. (tests-static): Likewise. (DEFAULT-LDFLAGS-tst-gmon-static): Removed. (tst-gmon-static-no-pie): New. (CFLAGS-tst-gmon-static-pie.c): Likewise. (CRT-tst-gmon-static-pie): Likewise. (tst-gmon-static-pie-ENV): Likewise. (tests-special): Likewise. ($(objpfx)tst-gmon-static-pie.out): Likewise. (clean-tst-gmon-static-pie-data): Likewise. ($(objpfx)tst-gmon-static-pie-gprof.out): Likewise. * gmon/tst-gmon-static-pie.c: New file. * manual/install.texi: Document --enable-static-pie. * sysdeps/generic/ldsodefs.h (_dl_relocate_static_pie): New. (_dl_get_dl_main_map): Likewise. * sysdeps/i386/configure.ac: Check if linker supports static PIE. * sysdeps/x86_64/configure.ac: Likewise. * sysdeps/i386/configure: Regenerated. * sysdeps/x86_64/configure: Likewise. * sysdeps/mips/Makefile (ASFLAGS-.o): Add $(pie-default). (ASFLAGS-.op): Likewise. --- ChangeLog | 72 +++++++++++++++++++++++++++++++++++++++++ INSTALL | 9 ++++++ Makeconfig | 38 +++++++++++++++++++--- NEWS | 9 ++++++ config.h.in | 3 ++ configure | 79 +++++++++++++++++++++++++++++++++++++++++++++ configure.ac | 29 +++++++++++++++++ csu/Makefile | 24 ++++++++++++-- csu/libc-start.c | 2 ++ csu/libc-tls.c | 6 ++-- csu/static-reloc.c | 26 +++++++++++++++ elf/Makefile | 7 ++-- elf/dl-reloc-static-pie.c | 52 +++++++++++++++++++++++++++++ elf/dl-support.c | 11 +++++++ elf/dynamic-link.h | 2 +- elf/get-dynamic-info.h | 6 ++-- gmon/Makefile | 25 +++++++++++++- gmon/tst-gmon-static-pie.c | 1 + manual/install.texi | 9 ++++++ sysdeps/generic/ldsodefs.h | 11 +++++++ sysdeps/i386/configure | 33 +++++++++++++++++++ sysdeps/i386/configure.ac | 23 +++++++++++++ sysdeps/mips/Makefile | 3 ++ sysdeps/x86_64/configure | 33 +++++++++++++++++++ sysdeps/x86_64/configure.ac | 25 ++++++++++++++ 25 files changed, 521 insertions(+), 17 deletions(-) create mode 100644 csu/static-reloc.c create mode 100644 elf/dl-reloc-static-pie.c create mode 100644 gmon/tst-gmon-static-pie.c (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 852043e521..e9f203fd37 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,75 @@ +2017-12-15 H.J. Lu + + [BZ #19574] + * INSTALL: Regenerated. + * Makeconfig (real-static-start-installed-name): New. + (pic-default): Updated for --enable-static-pie. + (pie-default): New for --enable-static-pie. + (default-pie-ldflag): Likewise. + (+link-static-before-libc): Replace $(DEFAULT-LDFLAGS-$(@F)) + with $(if $($(@F)-no-pie),$(no-pie-ldflag),$(default-pie-ldflag)). + Replace $(static-start-installed-name) with + $(real-static-start-installed-name). + (+prectorT): Updated for --enable-static-pie. + (+postctorT): Likewise. + (CFLAGS-.o): Add $(pie-default). + (CFLAGS-.op): Likewise. + * NEWS: Mention --enable-static-pie. + * config.h.in (ENABLE_STATIC_PIE): New. + * configure.ac (--enable-static-pie): New configure option. + (have-no-dynamic-linker): New LIBC_CONFIG_VAR. + (have-static-pie): Likewise. + Enable static PIE if linker supports --no-dynamic-linker. + (ENABLE_STATIC_PIE): New AC_DEFINE. + (enable-static-pie): New LIBC_CONFIG_VAR. + * configure: Regenerated. + * csu/Makefile (omit-deps): Add r$(start-installed-name) and + gr$(start-installed-name) for --enable-static-pie. + (extra-objs): Likewise. + (install-lib): Likewise. + (extra-objs): Add static-reloc.o and static-reloc.os + ($(objpfx)$(start-installed-name)): Also depend on + $(objpfx)static-reloc.o. + ($(objpfx)r$(start-installed-name)): New. + ($(objpfx)g$(start-installed-name)): Also depend on + $(objpfx)static-reloc.os. + ($(objpfx)gr$(start-installed-name)): New. + * csu/libc-start.c (LIBC_START_MAIN): Call _dl_relocate_static_pie + in libc.a. + * csu/libc-tls.c (__libc_setup_tls): Add main_map->l_addr to + initimage. + * csu/static-reloc.c: New file. + * elf/Makefile (routines): Add dl-reloc-static-pie. + (elide-routines.os): Likewise. + (DEFAULT-LDFLAGS-tst-tls1-static-non-pie): Removed. + (tst-tls1-static-non-pie-no-pie): New. + * elf/dl-reloc-static-pie.c: New file. + * elf/dl-support.c (_dl_get_dl_main_map): New function. + * elf/dynamic-link.h (ELF_DURING_STARTUP): Also check + STATIC_PIE_BOOTSTRAP. + * elf/get-dynamic-info.h (elf_get_dynamic_info): Likewise. + * gmon/Makefile (tests): Add tst-gmon-static-pie. + (tests-static): Likewise. + (DEFAULT-LDFLAGS-tst-gmon-static): Removed. + (tst-gmon-static-no-pie): New. + (CFLAGS-tst-gmon-static-pie.c): Likewise. + (CRT-tst-gmon-static-pie): Likewise. + (tst-gmon-static-pie-ENV): Likewise. + (tests-special): Likewise. + ($(objpfx)tst-gmon-static-pie.out): Likewise. + (clean-tst-gmon-static-pie-data): Likewise. + ($(objpfx)tst-gmon-static-pie-gprof.out): Likewise. + * gmon/tst-gmon-static-pie.c: New file. + * manual/install.texi: Document --enable-static-pie. + * sysdeps/generic/ldsodefs.h (_dl_relocate_static_pie): New. + (_dl_get_dl_main_map): Likewise. + * sysdeps/i386/configure.ac: Check if linker supports static PIE. + * sysdeps/x86_64/configure.ac: Likewise. + * sysdeps/i386/configure: Regenerated. + * sysdeps/x86_64/configure: Likewise. + * sysdeps/mips/Makefile (ASFLAGS-.o): Add $(pie-default). + (ASFLAGS-.op): Likewise. + 2017-12-15 Joseph Myers * io/Makefile (tst-open-tmpfile-ARGS): New variable. diff --git a/INSTALL b/INSTALL index 80306de35c..9a1404bd3c 100644 --- a/INSTALL +++ b/INSTALL @@ -86,6 +86,15 @@ will be used, and CFLAGS sets optimization options for the compiler. systems support shared libraries; you need ELF support and (currently) the GNU linker. +'--enable-static-pie' + Enable static position independent executable (static PIE) support. + Static PIE is similar to static executable, but can be loaded at + any address without help from a dynamic linker. All static + programs as well as static tests are built as static PIE, except + for those marked with no-pie. The resulting glibc can be used with + the GCC option, -static-pie, which is available with GCC 8 or + above, to create static PIE. + '--disable-profile' Don't build libraries with profiling information. You may want to use this option if you don't plan to do profiling. diff --git a/Makeconfig b/Makeconfig index 1346109ac0..99cc136bfa 100644 --- a/Makeconfig +++ b/Makeconfig @@ -352,6 +352,14 @@ ifndef static-start-installed-name static-start-installed-name = $(start-installed-name) endif +ifeq (yes,$(enable-static-pie)) +# Link with rcrt1.o, instead of crt1.o, to call _dl_relocate_static_pie +# to relocate static PIE. +real-static-start-installed-name = r$(static-start-installed-name) +else +real-static-start-installed-name = $(static-start-installed-name) +endif + ifeq (yesyes,$(build-shared)$(have-z-combreloc)) combreloc-LDFLAGS = -Wl,-z,combreloc LDFLAGS.so += $(combreloc-LDFLAGS) @@ -371,6 +379,20 @@ LDFLAGS.so += $(hashstyle-LDFLAGS) LDFLAGS-rtld += $(hashstyle-LDFLAGS) endif +ifeq (yes,$(enable-static-pie)) +pic-default = -DPIC +# Compile libc.a and libc_p.a with -fPIE/-fpie for static PIE. +pie-default = $(pie-ccflag) +ifeq (yes,$(have-static-pie)) +default-pie-ldflag = -static-pie +else +# Static PIE can't have dynamic relocations in read-only segments since +# static PIE is mapped into memory by kernel. --eh-frame-hdr is needed +# for PIE to support exception. +default-pie-ldflag = -Wl,-pie,--no-dynamic-linker,--eh-frame-hdr,-z,text +endif +endif + # If lazy relocations are disabled, add the -z now flag. Use # LDFLAGS-lib.so instead of LDFLAGS.so, to avoid adding the flag to # test modules. @@ -420,9 +442,9 @@ endif # Command for statically linking programs with the C library. ifndef +link-static +link-static-before-libc = $(CC) -nostdlib -nostartfiles -static -o $@ \ - $(DEFAULT-LDFLAGS-$(@F)) \ + $(if $($(@F)-no-pie),$(no-pie-ldflag),$(default-pie-ldflag)) \ $(sysdep-LDFLAGS) $(LDFLAGS) $(LDFLAGS-$(@F)) \ - $(firstword $(CRT-$(@F)) $(csu-objpfx)$(static-start-installed-name)) \ + $(firstword $(CRT-$(@F)) $(csu-objpfx)$(real-static-start-installed-name)) \ $(+preinit) $(+prectorT) \ $(filter-out $(addprefix $(csu-objpfx),start.o \ $(start-installed-name))\ @@ -637,8 +659,14 @@ endif +prectorS = `$(CC) $(sysdep-LDFLAGS) --print-file-name=crtbeginS.o` +postctorS = `$(CC) $(sysdep-LDFLAGS) --print-file-name=crtendS.o` # Variants of the two previous definitions for statically linking programs. +ifeq (yes,$(enable-static-pie)) +# Static PIE must use PIE variants. ++prectorT = $(+prectorS) ++postctorT = $(+postctorS) +else +prectorT = `$(CC) $(sysdep-LDFLAGS) --print-file-name=crtbeginT.o` +postctorT = `$(CC) $(sysdep-LDFLAGS) --print-file-name=crtend.o` +endif csu-objpfx = $(common-objpfx)csu/ elf-objpfx = $(common-objpfx)elf/ @@ -959,7 +987,8 @@ libtypes = $(foreach o,$(object-suffixes-for-libc),$(libtype$o)) all-object-suffixes := .o .os .oS object-suffixes := CPPFLAGS-.o = $(pic-default) -CFLAGS-.o = $(filter %frame-pointer,$(+cflags)) +# libc.a must be compiled with -fPIE/-fpie for static PIE. +CFLAGS-.o = $(filter %frame-pointer,$(+cflags)) $(pie-default) libtype.o := lib%.a object-suffixes += .o ifeq (yes,$(build-shared)) @@ -984,7 +1013,8 @@ ifeq (yes,$(build-profile)) all-object-suffixes += .op object-suffixes += .op CPPFLAGS-.op = -DPROF $(pic-default) -CFLAGS-.op = -pg +# libc_p.a must be compiled with -fPIE/-fpie for static PIE. +CFLAGS-.op = -pg $(pie-default) libtype.op = lib%_p.a endif diff --git a/NEWS b/NEWS index c5607c855f..61fed654d8 100644 --- a/NEWS +++ b/NEWS @@ -9,6 +9,15 @@ Version 2.27 Major new features: +* The GNU C Library can now be compiled with support for building static + PIE executables (See --enable-static-pie in INSTALL). These static PIE + exectuables are like static executables but can be loaded at any address + and provide additional security hardening benefits at the cost of some + memory and performance. When the library is built with --enable-static-pie + the resulting libc.a is usable with GCC 8 and above to create static PIE + executables using the GCC option '-static-pie'. This feature is currently + supported on i386, x86_64 and x32. + * Optimized x86-64 asin, atan2, exp, expf, log, pow, atan, sin, cosf, sinf and tan with FMA, contributed by Arjan van de Ven and H.J. Lu from Intel. diff --git a/config.h.in b/config.h.in index 3c91d597ff..d928e7dd86 100644 --- a/config.h.in +++ b/config.h.in @@ -238,6 +238,9 @@ /* Build glibc with tunables support. */ #define HAVE_TUNABLES 0 +/* Define if static PIE is enabled. */ +#define ENABLE_STATIC_PIE 0 + /* Some compiler options may now allow to use ebp in __asm__ (used mainly in i386 6 argument syscall issue). */ #define CAN_USE_REGISTER_ASM_EBP 0 diff --git a/configure b/configure index 4a85706862..9da9fb2736 100755 --- a/configure +++ b/configure @@ -763,6 +763,7 @@ with_default_link enable_sanity_checks enable_shared enable_profile +enable_static_pie enable_timezone_tools enable_hardcoded_path_in_tests enable_stackguard_randomization @@ -1417,6 +1418,8 @@ Optional Features: in special situations) [default=yes] --enable-shared build shared library [default=yes if GNU ld] --enable-profile build profiled library [default=no] + --enable-static-pie enable static PIE support and use it in the + testsuite [default=no] --disable-timezone-tools do not install timezone tools [default=install] --enable-hardcoded-path-in-tests @@ -3350,6 +3353,13 @@ else profile=no fi +# Check whether --enable-static-pie was given. +if test "${enable_static_pie+set}" = set; then : + enableval=$enable_static_pie; static_pie=$enableval +else + static_pie=no +fi + # Check whether --enable-timezone-tools was given. if test "${enable_timezone_tools+set}" = set; then : enableval=$enable_timezone_tools; enable_timezone_tools=$enableval @@ -5806,6 +5816,62 @@ fi $as_echo "$libc_linker_feature" >&6; } +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for linker that supports --no-dynamic-linker" >&5 +$as_echo_n "checking for linker that supports --no-dynamic-linker... " >&6; } +libc_linker_feature=no +if test x"$gnu_ld" = x"yes"; then + libc_linker_check=`$LD -v --help 2>/dev/null | grep "\--no-dynamic-linker"` + if test -n "$libc_linker_check"; then + cat > conftest.c <&5 + (eval $ac_try) 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; } + then + libc_linker_feature=yes + fi + rm -f conftest* + fi +fi +if test $libc_linker_feature = yes; then + libc_cv_no_dynamic_linker=yes +else + libc_cv_no_dynamic_linker=no +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_linker_feature" >&5 +$as_echo "$libc_linker_feature" >&6; } +config_vars="$config_vars +have-no-dynamic-linker = $libc_cv_no_dynamic_linker" + +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for -static-pie" >&5 +$as_echo_n "checking for -static-pie... " >&6; } +if ${libc_cv_static_pie+:} false; then : + $as_echo_n "(cached) " >&6 +else + if { ac_try='${CC-cc} -static-pie -xc /dev/null -S -o /dev/null' + { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 + (eval $ac_try) 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; }; then : + libc_cv_static_pie=yes +else + libc_cv_static_pie=no +fi + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_static_pie" >&5 +$as_echo "$libc_cv_static_pie" >&6; } +config_vars="$config_vars +have-static-pie = $libc_cv_static_pie" + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -fpie" >&5 $as_echo_n "checking for -fpie... " >&6; } if ${libc_cv_fpie+:} false; then : @@ -6705,6 +6771,19 @@ $as_echo "$libc_cv_pie_default" >&6; } libc_cv_multidir=`${CC-cc} $CFLAGS $CPPFLAGS -print-multi-directory` +if test "$static_pie" = yes; then + # The linker must support --no-dynamic-linker. + if test "$libc_cv_no_dynamic_linker" != yes; then + as_fn_error $? "linker support for --no-dynamic-linker needed" "$LINENO" 5 + fi + # Default to PIE. + libc_cv_pie_default=yes + $as_echo "#define ENABLE_STATIC_PIE 1" >>confdefs.h + +fi +config_vars="$config_vars +enable-static-pie = $static_pie" + diff --git a/configure.ac b/configure.ac index edf662715b..8e4006f0a9 100644 --- a/configure.ac +++ b/configure.ac @@ -170,6 +170,11 @@ AC_ARG_ENABLE([profile], [build profiled library @<:@default=no@:>@]), [profile=$enableval], [profile=no]) +AC_ARG_ENABLE([static-pie], + AC_HELP_STRING([--enable-static-pie], + [enable static PIE support and use it in the testsuite @<:@default=no@:>@]), + [static_pie=$enableval], + [static_pie=no]) AC_ARG_ENABLE([timezone-tools], AC_HELP_STRING([--disable-timezone-tools], [do not install timezone tools @<:@default=install@:>@]), @@ -1287,6 +1292,19 @@ LIBC_LINKER_FEATURE([-z execstack], [-Wl,-z,execstack], [libc_cv_z_execstack=yes], [libc_cv_z_execstack=no]) AC_SUBST(libc_cv_z_execstack) +LIBC_LINKER_FEATURE([--no-dynamic-linker], + [-Wl,--no-dynamic-linker], + [libc_cv_no_dynamic_linker=yes], + [libc_cv_no_dynamic_linker=no]) +LIBC_CONFIG_VAR([have-no-dynamic-linker], [$libc_cv_no_dynamic_linker]) + +AC_CACHE_CHECK(for -static-pie, libc_cv_static_pie, [dnl +LIBC_TRY_CC_OPTION([-static-pie], + [libc_cv_static_pie=yes], + [libc_cv_static_pie=no]) +]) +LIBC_CONFIG_VAR([have-static-pie], [$libc_cv_static_pie]) + AC_CACHE_CHECK(for -fpie, libc_cv_fpie, [dnl LIBC_TRY_CC_OPTION([-fpie], [libc_cv_fpie=yes], [libc_cv_fpie=no]) ]) @@ -1794,6 +1812,17 @@ AC_SUBST(libc_cv_pie_default) libc_cv_multidir=`${CC-cc} $CFLAGS $CPPFLAGS -print-multi-directory` AC_SUBST(libc_cv_multidir) +if test "$static_pie" = yes; then + # The linker must support --no-dynamic-linker. + if test "$libc_cv_no_dynamic_linker" != yes; then + AC_MSG_ERROR([linker support for --no-dynamic-linker needed]) + fi + # Default to PIE. + libc_cv_pie_default=yes + AC_DEFINE(ENABLE_STATIC_PIE) +fi +LIBC_CONFIG_VAR([enable-static-pie], [$static_pie]) + AC_SUBST(profile) AC_SUBST(static_nss) diff --git a/csu/Makefile b/csu/Makefile index e42a32b3eb..86b95a9759 100644 --- a/csu/Makefile +++ b/csu/Makefile @@ -37,7 +37,9 @@ extra-objs = start.o \ S$(start-installed-name) omit-deps = $(patsubst %.o,%,$(start-installed-name) g$(start-installed-name) \ b$(start-installed-name) $(csu-dummies) \ - S$(start-installed-name)) + S$(start-installed-name) \ + r$(start-installed-name) \ + gr$(start-installed-name)) install-lib = $(start-installed-name) g$(start-installed-name) $(csu-dummies) # No tests are allowed in the csu/ subdirectory because the startup @@ -60,10 +62,17 @@ extra-objs += gmon-start.o endif ifneq ($(start-installed-name),$(static-start-installed-name)) +# FIXME: Only Hurd defines static-start-installed-name. Hurd needs to +# provide special rules to support static PIE. extra-objs += $(static-start-installed-name) g$(static-start-installed-name) omit-deps += $(patsubst %.o,%,$(static-start-installed-name) \ g$(static-start-installed-name)) install-lib += $(static-start-installed-name) g$(static-start-installed-name) +else +ifeq (yes,$(enable-static-pie)) +extra-objs += r$(start-installed-name) gr$(start-installed-name) +install-lib += r$(start-installed-name) gr$(start-installed-name) +endif endif before-compile += $(objpfx)abi-tag.h @@ -82,7 +91,10 @@ multilib-extra-objs = $(addprefix $(multidir)/, $(install-lib)) extra-objs += $(multilib-extra-objs) endif -extra-objs += abi-note.o init.o +extra-objs += abi-note.o init.o static-reloc.o +ifeq (yes,$(build-shared)) +extra-objs += static-reloc.os +endif asm-CPPFLAGS += -I$(objpfx). # Enable unwinding so backtrace unwinds to __libc_start_main @@ -101,6 +113,9 @@ ifndef start-installed-name-rule # We link the ELF startfile along with a SHT_NOTE section indicating # the kernel ABI the binaries linked with this library will require. $(objpfx)$(start-installed-name): $(objpfx)start.o $(objpfx)abi-note.o \ + $(objpfx)init.o $(objpfx)static-reloc.o + $(link-relocatable) +$(objpfx)r$(start-installed-name): $(objpfx)start.o $(objpfx)abi-note.o \ $(objpfx)init.o $(link-relocatable) $(objpfx)S$(start-installed-name): $(objpfx)start.os $(objpfx)abi-note.o \ @@ -113,7 +128,10 @@ endif # to turn on profiling code at startup. ifeq (yes,$(build-shared)) $(objpfx)g$(start-installed-name): \ - $(objpfx)g%: $(objpfx)S% $(objpfx)gmon-start.os + $(objpfx)g%: $(objpfx)S% $(objpfx)gmon-start.os $(objpfx)static-reloc.os + $(link-relocatable) +$(objpfx)gr$(start-installed-name): \ + $(objpfx)gr%: $(objpfx)r% $(objpfx)gmon-start.o $(link-relocatable) ifneq ($(start-installed-name),$(static-start-installed-name)) $(objpfx)g$(static-start-installed-name): \ diff --git a/csu/libc-start.c b/csu/libc-start.c index 24c63be02f..34dd125260 100644 --- a/csu/libc-start.c +++ b/csu/libc-start.c @@ -141,6 +141,8 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), __libc_multiple_libcs = &_dl_starting_up && !_dl_starting_up; #ifndef SHARED + _dl_relocate_static_pie (); + char **ev = &argv[argc + 1]; __environ = ev; diff --git a/csu/libc-tls.c b/csu/libc-tls.c index 00138eb43a..1f8ddaf543 100644 --- a/csu/libc-tls.c +++ b/csu/libc-tls.c @@ -114,6 +114,8 @@ __libc_setup_tls (void) size_t tcb_offset; const ElfW(Phdr) *phdr; + struct link_map *main_map = GL(dl_ns)[LM_ID_BASE]._ns_loaded; + /* Look through the TLS segment if there is any. */ if (_dl_phdr != NULL) for (phdr = _dl_phdr; phdr < &_dl_phdr[_dl_phnum]; ++phdr) @@ -122,7 +124,7 @@ __libc_setup_tls (void) /* Remember the values we need. */ memsz = phdr->p_memsz; filesz = phdr->p_filesz; - initimage = (void *) phdr->p_vaddr; + initimage = (void *) phdr->p_vaddr + main_map->l_addr; align = phdr->p_align; if (phdr->p_align > max_align) max_align = phdr->p_align; @@ -163,8 +165,6 @@ __libc_setup_tls (void) _dl_static_dtv[0].counter = (sizeof (_dl_static_dtv) / sizeof (_dl_static_dtv[0])) - 2; // _dl_static_dtv[1].counter = 0; would be needed if not already done - struct link_map *main_map = GL(dl_ns)[LM_ID_BASE]._ns_loaded; - /* Initialize the TLS block. */ #if TLS_TCB_AT_TP _dl_static_dtv[2].pointer.val = ((char *) tlsblock + tcb_offset diff --git a/csu/static-reloc.c b/csu/static-reloc.c new file mode 100644 index 0000000000..37be72c8ea --- /dev/null +++ b/csu/static-reloc.c @@ -0,0 +1,26 @@ +/* Special startup support for non-PIE static executables. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if ENABLE_STATIC_PIE +#include + +void +_dl_relocate_static_pie (void) +{ +} +#endif diff --git a/elf/Makefile b/elf/Makefile index 8563555079..47c3d23ed8 100644 --- a/elf/Makefile +++ b/elf/Makefile @@ -24,7 +24,8 @@ include ../Makeconfig headers = elf.h bits/elfclass.h link.h bits/link.h routines = $(all-dl-routines) dl-support dl-iteratephdr \ dl-addr dl-addr-obj enbl-secure dl-profstub \ - dl-origin dl-libc dl-sym dl-sysdep dl-error + dl-origin dl-libc dl-sym dl-sysdep dl-error \ + dl-reloc-static-pie # The core dynamic linking functions are in libc for the static and # profiled libraries. @@ -52,7 +53,7 @@ endif all-dl-routines = $(dl-routines) $(sysdep-dl-routines) # But they are absent from the shared libc, because that code is in ld.so. elide-routines.os = $(all-dl-routines) dl-support enbl-secure dl-origin \ - dl-sysdep dl-exception + dl-sysdep dl-exception dl-reloc-static-pie shared-only-routines += dl-caller # ld.so uses those routines, plus some special stuff for being the program @@ -153,7 +154,7 @@ tests-static-internal := tst-tls1-static tst-tls2-static \ tst-tls1-static-non-pie CRT-tst-tls1-static-non-pie := $(csu-objpfx)crt1.o -DEFAULT-LDFLAGS-tst-tls1-static-non-pie = $(no-pie-ldflag) +tst-tls1-static-non-pie-no-pie = yes tests := tst-tls9 tst-leaks1 \ tst-array1 tst-array2 tst-array3 tst-array4 tst-array5 \ diff --git a/elf/dl-reloc-static-pie.c b/elf/dl-reloc-static-pie.c new file mode 100644 index 0000000000..6e43aea76a --- /dev/null +++ b/elf/dl-reloc-static-pie.c @@ -0,0 +1,52 @@ +/* Support for relocating static PIE. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if ENABLE_STATIC_PIE +#include +#include +#include "dynamic-link.h" + +/* Relocate static executable with PIE. */ + +void +_dl_relocate_static_pie (void) +{ + struct link_map *main_map = _dl_get_dl_main_map (); + +# define STATIC_PIE_BOOTSTRAP +# define BOOTSTRAP_MAP (main_map) +# define RESOLVE_MAP(sym, version, flags) BOOTSTRAP_MAP +# include "dynamic-link.h" + + /* Figure out the run-time load address of static PIE. */ + main_map->l_addr = elf_machine_load_address (); + + /* Read our own dynamic section and fill in the info array. */ + main_map->l_ld = ((void *) main_map->l_addr + elf_machine_dynamic ()); + elf_get_dynamic_info (main_map, NULL); + +# ifdef ELF_MACHINE_BEFORE_RTLD_RELOC + ELF_MACHINE_BEFORE_RTLD_RELOC (main_map->l_info); +# endif + + /* Relocate ourselves so we can do normal function calls and + data access using the global offset table. */ + ELF_DYNAMIC_RELOCATE (main_map, 0, 0, 0); + main_map->l_relocated = 1; +} +#endif diff --git a/elf/dl-support.c b/elf/dl-support.c index 235d3a7f46..b9fc1a66fe 100644 --- a/elf/dl-support.c +++ b/elf/dl-support.c @@ -385,3 +385,14 @@ _dl_non_dynamic_init (void) #ifdef DL_SYSINFO_IMPLEMENTATION DL_SYSINFO_IMPLEMENTATION #endif + +#if ENABLE_STATIC_PIE +/* Since relocation to hidden _dl_main_map causes relocation overflow on + aarch64, a function is used to get the address of _dl_main_map. */ + +struct link_map * +_dl_get_dl_main_map (void) +{ + return &_dl_main_map; +} +#endif diff --git a/elf/dynamic-link.h b/elf/dynamic-link.h index ebea7567cd..6278649711 100644 --- a/elf/dynamic-link.h +++ b/elf/dynamic-link.h @@ -94,7 +94,7 @@ elf_machine_lazy_rel (struct link_map *map, #ifdef RESOLVE_MAP -# ifdef RTLD_BOOTSTRAP +# if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP # define ELF_DURING_STARTUP (1) # else # define ELF_DURING_STARTUP (0) diff --git a/elf/get-dynamic-info.h b/elf/get-dynamic-info.h index 7525c3a5b2..eb26d23649 100644 --- a/elf/get-dynamic-info.h +++ b/elf/get-dynamic-info.h @@ -38,7 +38,7 @@ elf_get_dynamic_info (struct link_map *l, ElfW(Dyn) *temp) typedef Elf64_Xword d_tag_utype; #endif -#ifndef RTLD_BOOTSTRAP +#if !defined RTLD_BOOTSTRAP && !defined STATIC_PIE_BOOTSTRAP if (dyn == NULL) return; #endif @@ -139,9 +139,11 @@ elf_get_dynamic_info (struct link_map *l, ElfW(Dyn) *temp) /* Only the bind now flags are allowed. */ assert (info[VERSYMIDX (DT_FLAGS_1)] == NULL || (info[VERSYMIDX (DT_FLAGS_1)]->d_un.d_val & ~DF_1_NOW) == 0); + /* Flags must not be set for ld.so. */ assert (info[DT_FLAGS] == NULL || (info[DT_FLAGS]->d_un.d_val & ~DF_BIND_NOW) == 0); - /* Flags must not be set for ld.so. */ +#endif +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP assert (info[DT_RUNPATH] == NULL); assert (info[DT_RPATH] == NULL); #else diff --git a/gmon/Makefile b/gmon/Makefile index 89ab3fc7da..29e746723e 100644 --- a/gmon/Makefile +++ b/gmon/Makefile @@ -39,6 +39,10 @@ tests-static += tst-gmon-static ifeq (yesyes,$(have-fpie)$(build-shared)) tests += tst-gmon-pie tests-pie += tst-gmon-pie +ifeq (yes,$(enable-static-pie)) +tests += tst-gmon-static-pie +tests-static += tst-gmon-static-pie +endif endif # The mcount code won't work without a frame pointer. @@ -54,7 +58,7 @@ endif CFLAGS-tst-gmon-static.c := $(PIE-ccflag) -fno-omit-frame-pointer -pg CRT-tst-gmon-static := $(csu-objpfx)gcrt1.o -DEFAULT-LDFLAGS-tst-gmon-static = $(no-pie-ldflag) +tst-gmon-static-no-pie = yes tst-gmon-static-ENV := GMON_OUT_PREFIX=$(objpfx)tst-gmon-static.data ifeq ($(run-built-tests),yes) tests-special += $(objpfx)tst-gmon-static-gprof.out @@ -67,6 +71,15 @@ ifeq ($(run-built-tests),yes) tests-special += $(objpfx)tst-gmon-pie-gprof.out endif +ifeq (yes,$(enable-static-pie)) +CFLAGS-tst-gmon-static-pie.c := $(PIE-ccflag) -fno-omit-frame-pointer -pg +CRT-tst-gmon-static-pie := $(csu-objpfx)grcrt1.o +tst-gmon-static-pie-ENV := GMON_OUT_PREFIX=$(objpfx)tst-gmon-static-pie.data +ifeq ($(run-built-tests),yes) +tests-special += $(objpfx)tst-gmon-static-pie-gprof.out +endif +endif + include ../Rules @@ -111,3 +124,13 @@ clean-tst-gmon-pie-data: $(objpfx)tst-gmon-pie-gprof.out: tst-gmon-gprof.sh $(objpfx)tst-gmon-pie.out $(SHELL) $< $(GPROF) $(objpfx)tst-gmon-pie $(objpfx)tst-gmon-pie.data.* > $@; \ $(evaluate-test) + +$(objpfx)tst-gmon-static-pie.out: clean-tst-gmon-static-pie-data +clean-tst-gmon-static-pie-data: + rm -f $(objpfx)tst-gmon-static-pie.data.* + +$(objpfx)tst-gmon-static-pie-gprof.out: tst-gmon-static-gprof.sh \ + $(objpfx)tst-gmon-static-pie.out + $(SHELL) $< $(GPROF) $(objpfx)tst-gmon-static-pie \ + $(objpfx)tst-gmon-static-pie.data.* > $@; \ + $(evaluate-test) diff --git a/gmon/tst-gmon-static-pie.c b/gmon/tst-gmon-static-pie.c new file mode 100644 index 0000000000..1eef2583b6 --- /dev/null +++ b/gmon/tst-gmon-static-pie.c @@ -0,0 +1 @@ +#include "tst-gmon.c" diff --git a/manual/install.texi b/manual/install.texi index e81f1c50c3..fb956b5d6a 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -116,6 +116,15 @@ Don't build shared libraries even if it is possible. Not all systems support shared libraries; you need ELF support and (currently) the GNU linker. +@item --enable-static-pie +Enable static position independent executable (static PIE) support. +Static PIE is similar to static executable, but can be loaded at any +address without help from a dynamic linker. All static programs as +well as static tests are built as static PIE, except for those marked +with no-pie. The resulting glibc can be used with the GCC option, +-static-pie, which is available with GCC 8 or above, to create static +PIE. + @item --disable-profile Don't build libraries with profiling information. You may want to use this option if you don't plan to do profiling. diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h index 7a65dc641c..196513851f 100644 --- a/sysdeps/generic/ldsodefs.h +++ b/sysdeps/generic/ldsodefs.h @@ -1051,6 +1051,17 @@ extern void _dl_determine_tlsoffset (void) attribute_hidden; stack protector, among other things). */ void __libc_setup_tls (void); +# if ENABLE_STATIC_PIE +/* Relocate static executable with PIE. */ +extern void _dl_relocate_static_pie (void) attribute_hidden; + +/* Get a pointer to _dl_main_map. */ +extern struct link_map * _dl_get_dl_main_map (void) + __attribute__ ((visibility ("hidden"))); +# else +# define _dl_relocate_static_pie() +# endif + /* Initialization of libpthread for statically linked applications. If libpthread is not linked in, this is an empty function. */ void __pthread_initialize_minimal (void) weak_function; diff --git a/sysdeps/i386/configure b/sysdeps/i386/configure index 4cf968d8bc..90c63caf35 100644 --- a/sysdeps/i386/configure +++ b/sysdeps/i386/configure @@ -50,6 +50,39 @@ fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_compiler_builtin_inlined" >&5 $as_echo "$libc_compiler_builtin_inlined" >&6; } +if test "$static_pie" = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for linker static PIE support" >&5 +$as_echo_n "checking for linker static PIE support... " >&6; } +if ${libc_cv_ld_static_pie+:} false; then : + $as_echo_n "(cached) " >&6 +else + cat > conftest.s <<\EOF + .text + .global _start +_start: + movl _start@GOT(%ebx), %eax +EOF + libc_cv_pie_option="-Wl,-pie" + libc_cv_ld_static_pie=no + if { ac_try='${CC-cc} $CFLAGS $CPPFLAGS $LDFLAGS -nostartfiles -nostdlib $no_ssp $libc_cv_pie_option -o conftest conftest.s 1>&5' + { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 + (eval $ac_try) 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; }; then + if $READELF -r conftest | grep 'There are no relocations in this file.' > /dev/null; then + libc_cv_ld_static_pie=yes + fi + fi +rm -f conftest* +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_ld_static_pie" >&5 +$as_echo "$libc_cv_ld_static_pie" >&6; } + if test "$libc_cv_ld_static_pie" != yes; then + as_fn_error $? "linker support for static PIE needed" "$LINENO" 5 + fi +fi + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for Intel MPX support" >&5 $as_echo_n "checking for Intel MPX support... " >&6; } if ${libc_cv_asm_mpx+:} false; then : diff --git a/sysdeps/i386/configure.ac b/sysdeps/i386/configure.ac index b598b120bc..6d2068d2b3 100644 --- a/sysdeps/i386/configure.ac +++ b/sysdeps/i386/configure.ac @@ -30,6 +30,29 @@ LIBC_COMPILER_BUILTIN_INLINED( *** Please use host i786, i686, i586, or i486. *** For example: /source/glibc/configure CFLAGS='-O2 -march=i686' ...])]) +dnl Check if linker can convert "movl main@GOT(%ebx), %eax" to +dnl "leal main@GOTOFF(%ebx), %eax" for static PIE. +if test "$static_pie" = yes; then + AC_CACHE_CHECK(for linker static PIE support, libc_cv_ld_static_pie, [dnl +cat > conftest.s <<\EOF + .text + .global _start +_start: + movl _start@GOT(%ebx), %eax +EOF + libc_cv_pie_option="-Wl,-pie" + libc_cv_ld_static_pie=no + if AC_TRY_COMMAND(${CC-cc} $CFLAGS $CPPFLAGS $LDFLAGS -nostartfiles -nostdlib $no_ssp $libc_cv_pie_option -o conftest conftest.s 1>&AS_MESSAGE_LOG_FD); then + if $READELF -r conftest | grep 'There are no relocations in this file.' > /dev/null; then + libc_cv_ld_static_pie=yes + fi + fi +rm -f conftest*]) + if test "$libc_cv_ld_static_pie" != yes; then + AC_MSG_ERROR([linker support for static PIE needed]) + fi +fi + dnl Check whether asm supports Intel MPX AC_CACHE_CHECK(for Intel MPX support, libc_cv_asm_mpx, [dnl cat > conftest.s <<\EOF diff --git a/sysdeps/mips/Makefile b/sysdeps/mips/Makefile index fd891ddf09..7ac6fa5031 100644 --- a/sysdeps/mips/Makefile +++ b/sysdeps/mips/Makefile @@ -23,6 +23,9 @@ CPPFLAGS-crtn.S += $(pic-ccflag) endif ASFLAGS-.os += $(pic-ccflag) +# libc.a and libc_p.a must be compiled with -fPIE/-fpie for static PIE. +ASFLAGS-.o += $(pie-default) +ASFLAGS-.op += $(pie-default) ifeq ($(subdir),elf) ifneq ($(o32-fpabi),) diff --git a/sysdeps/x86_64/configure b/sysdeps/x86_64/configure index efef46b1b7..8674d14569 100644 --- a/sysdeps/x86_64/configure +++ b/sysdeps/x86_64/configure @@ -85,6 +85,39 @@ if test x"$build_mathvec" = xnotset; then build_mathvec=yes fi +if test "$static_pie" = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for linker static PIE support" >&5 +$as_echo_n "checking for linker static PIE support... " >&6; } +if ${libc_cv_ld_static_pie+:} false; then : + $as_echo_n "(cached) " >&6 +else + cat > conftest.s <<\EOF + .text + .global _start + .weak foo +_start: + leaq foo(%rip), %rax +EOF + libc_cv_pie_option="-Wl,-pie" + if { ac_try='${CC-cc} $CFLAGS $CPPFLAGS $LDFLAGS -nostartfiles -nostdlib $no_ssp $libc_cv_pie_option -o conftest conftest.s 1>&5' + { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 + (eval $ac_try) 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; }; then + libc_cv_ld_static_pie=yes + else + libc_cv_ld_static_pie=no + fi +rm -f conftest* +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_ld_static_pie" >&5 +$as_echo "$libc_cv_ld_static_pie" >&6; } + if test "$libc_cv_ld_static_pie" != yes; then + as_fn_error $? "linker support for static PIE needed" "$LINENO" 5 + fi +fi + $as_echo "#define PI_STATIC_AND_HIDDEN 1" >>confdefs.h diff --git a/sysdeps/x86_64/configure.ac b/sysdeps/x86_64/configure.ac index fa86e953ee..b7d2c0124f 100644 --- a/sysdeps/x86_64/configure.ac +++ b/sysdeps/x86_64/configure.ac @@ -44,6 +44,31 @@ if test x"$build_mathvec" = xnotset; then build_mathvec=yes fi +dnl Check if linker supports static PIE with the fix for +dnl +dnl https://sourceware.org/bugzilla/show_bug.cgi?id=21782 +dnl +if test "$static_pie" = yes; then + AC_CACHE_CHECK(for linker static PIE support, libc_cv_ld_static_pie, [dnl +cat > conftest.s <<\EOF + .text + .global _start + .weak foo +_start: + leaq foo(%rip), %rax +EOF + libc_cv_pie_option="-Wl,-pie" + if AC_TRY_COMMAND(${CC-cc} $CFLAGS $CPPFLAGS $LDFLAGS -nostartfiles -nostdlib $no_ssp $libc_cv_pie_option -o conftest conftest.s 1>&AS_MESSAGE_LOG_FD); then + libc_cv_ld_static_pie=yes + else + libc_cv_ld_static_pie=no + fi +rm -f conftest*]) + if test "$libc_cv_ld_static_pie" != yes; then + AC_MSG_ERROR([linker support for static PIE needed]) + fi +fi + dnl It is always possible to access static and hidden symbols in an dnl position independent way. AC_DEFINE(PI_STATIC_AND_HIDDEN) -- cgit v1.2.3 From 126adc89d8a32193df075ce665e76ad95ebd0557 Mon Sep 17 00:00:00 2001 From: "H.J. Lu" Date: Tue, 19 Dec 2017 13:51:01 -0800 Subject: Document that --enable-static-pie implies PIE To build static PIE, all .o files are compiled with -fPIE. Since --enable-static-pie is designed to provide additional security hardening benefits, it also implies that glibc programs and tests are created as dynamic position independent executables (PIE) by default for better security hardening. Reviewed-by: Jonathan Nieder * manual/install.texi: Document that --enable-static-pie implies PIE. * INSTALL: Regenerated. --- ChangeLog | 6 ++++++ INSTALL | 4 +++- manual/install.texi | 3 ++- 3 files changed, 11 insertions(+), 2 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index d194a73592..9d567b20bf 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2017-12-19 H.J. Lu + + * manual/install.texi: Document that --enable-static-pie + implies PIE. + * INSTALL: Regenerated. + 2017-12-19 Bernd Edlinger [BZ #21309] diff --git a/INSTALL b/INSTALL index 9a1404bd3c..42508e69dc 100644 --- a/INSTALL +++ b/INSTALL @@ -93,7 +93,9 @@ will be used, and CFLAGS sets optimization options for the compiler. programs as well as static tests are built as static PIE, except for those marked with no-pie. The resulting glibc can be used with the GCC option, -static-pie, which is available with GCC 8 or - above, to create static PIE. + above, to create static PIE. This option also implies that glibc + programs and tests are created as dynamic position independent + executables (PIE) by default. '--disable-profile' Don't build libraries with profiling information. You may want to diff --git a/manual/install.texi b/manual/install.texi index fb956b5d6a..50e6c35050 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -123,7 +123,8 @@ address without help from a dynamic linker. All static programs as well as static tests are built as static PIE, except for those marked with no-pie. The resulting glibc can be used with the GCC option, -static-pie, which is available with GCC 8 or above, to create static -PIE. +PIE. This option also implies that glibc programs and tests are created +as dynamic position independent executables (PIE) by default. @item --disable-profile Don't build libraries with profiling information. You may want to use -- cgit v1.2.3 From 93493119648db44af0b8170588e02ad273ec86a4 Mon Sep 17 00:00:00 2001 From: "Dmitry V. Levin" Date: Wed, 31 Jan 2018 14:20:10 +0000 Subject: Update information about the newest versions of tools used to build glibc * manual/install.texi (Tools for Compilation): Update the newest versions of gcc, binutils, texinfo, gawk, bison, and sed. * INSTALL: Regenerated. --- ChangeLog | 6 ++++++ INSTALL | 13 +++++++------ manual/install.texi | 13 +++++++------ 3 files changed, 20 insertions(+), 12 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 06250f9007..b2cec4fea2 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2018-01-31 Dmitry V. Levin + + * manual/install.texi (Tools for Compilation): Update the newest + versions of gcc, binutils, texinfo, gawk, bison, and sed. + * INSTALL: Regenerated. + 2018-01-30 Samuel Thibault Date: Wed, 31 Jan 2018 23:13:00 +0000 Subject: Fix typo in the previous commit The version of GCC was 7.3, not 7.3.1. --- INSTALL | 2 +- manual/install.texi | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'INSTALL') diff --git a/INSTALL b/INSTALL index 68b9252b74..64dec3473e 100644 --- a/INSTALL +++ b/INSTALL @@ -419,7 +419,7 @@ build the GNU C Library: GCC 4.9 or higher is required. In general it is recommended to use the newest version of the compiler that is known to work for building the GNU C Library, as newer compilers usually produce - better code. As of release time, GCC 7.3.1 is the newest compiler + better code. As of release time, GCC 7.3 is the newest compiler verified to work to build the GNU C Library. For PowerPC 64-bits little-endian (powerpc64le), GCC 6.2 or higher diff --git a/manual/install.texi b/manual/install.texi index efcd98636f..43dd8d6b81 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -465,7 +465,7 @@ GCC 4.9 or newer GCC 4.9 or higher is required. In general it is recommended to use the newest version of the compiler that is known to work for building @theglibc{}, as newer compilers usually produce better code. As of -release time, GCC 7.3.1 is the newest compiler verified to work to build +release time, GCC 7.3 is the newest compiler verified to work to build @theglibc{}. For PowerPC 64-bits little-endian (powerpc64le), GCC 6.2 or higher is -- cgit v1.2.3 From db9881ecd7e7278af3e6bb252a0b3015e275d7bd Mon Sep 17 00:00:00 2001 From: Joseph Myers Date: Thu, 15 Feb 2018 23:48:47 +0000 Subject: Document use of CC and CFLAGS in more detail (bug 20980, bug 21234). There are some bug reports from people setting CFLAGS not including a -O option and then being confused when the build fails. This patch addresses this by documenting the proper use of CC and CFLAGS in more detail - saying what options should go where and specifying the requirement to compile with optimization. The previous text incorrectly used @var markup with CC and CFLAGS. The correct markup for environment variables is @env, but it's also the case that passing such variables explicitly on the configure command line is preferred to passing them in the environment, so this patch changes the documentation to describe passing them on the command line (and uses @code). In many cases putting options in the wrong place may in fact work, but I believe what I've specified is the correct rule for which options to put where. [BZ #20980] [BZ #21234] * manual/install.texi (Configuring and compiling): Describe passing CC and CFLAGS on configure command line, not as environment variables. Use @code markup on those variables. Specify what options go in CC and what go in CFLAGS. Note the requirement to compile with optimization. * INSTALL: Regenerated. --- ChangeLog | 9 +++++++++ INSTALL | 19 ++++++++++++++----- manual/install.texi | 25 ++++++++++++++++++------- 3 files changed, 41 insertions(+), 12 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 9bc1c16d2a..c5696dbe36 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,14 @@ 2018-02-15 Joseph Myers + [BZ #20980] + [BZ #21234] + * manual/install.texi (Configuring and compiling): Describe + passing CC and CFLAGS on configure command line, not as + environment variables. Use @code markup on those variables. + Specify what options go in CC and what go in CFLAGS. Note the + requirement to compile with optimization. + * INSTALL: Regenerated. + [BZ #18124] * sysdeps/hppa/bsd-setjmp.S: Include . (setjmp): Use HIDDEN_JUMPTARGET with __sigsetjmp. diff --git a/INSTALL b/INSTALL index 64dec3473e..052b1b6f89 100644 --- a/INSTALL +++ b/INSTALL @@ -36,9 +36,18 @@ normal setting to install as the standard system library is '--prefix=/usr' for GNU/Linux systems and '--prefix=' (an empty prefix) for GNU/Hurd systems. - It may also be useful to set the CC and CFLAGS variables in the -environment when running 'configure'. CC selects the C compiler that -will be used, and CFLAGS sets optimization options for the compiler. + It may also be useful to pass 'CC=COMPILER' and 'CFLAGS=FLAGS' +arguments to 'configure'. 'CC' selects the C compiler that will be +used, and 'CFLAGS' sets optimization options for the compiler. Any +compiler options required for all compilations, such as options +selecting an ABI or a processor for which to generate code, should be +included in 'CC'. Options that may be overridden by the GNU C Library +build system for particular files, such as for optimization and +debugging, should go in 'CFLAGS'. The default value of 'CFLAGS' is '-g +-O2', and the GNU C Library cannot be compiled without optimization, so +if 'CFLAGS' is specified it must enable optimization. For example: + + $ ../glibc-VERSION/configure CC="gcc -m32" CFLAGS="-O3" The following list describes all of the available options for 'configure': @@ -210,7 +219,7 @@ will be used, and CFLAGS sets optimization options for the compiler. but you want to compile a library for 586es, give '--host=i586-pc-linux-gnu' or just '--host=i586-linux' and add the appropriate compiler flags ('-mcpu=i586' will do the trick) to - CFLAGS. + 'CC'. If you specify just '--build', 'configure' will get confused. @@ -304,7 +313,7 @@ makefiles. setting a few variables in 'configparms'. Set 'CC' to the cross-compiler for the target you configured the library for; it is important to use this same 'CC' value when running 'configure', like -this: 'CC=TARGET-gcc configure TARGET'. Set 'BUILD_CC' to the compiler +this: 'configure TARGET CC=TARGET-gcc'. Set 'BUILD_CC' to the compiler to use for programs run on the build system as part of compiling the library. You may need to set 'AR' to cross-compiling versions of 'ar' if the native tools are not configured to work with object files for the diff --git a/manual/install.texi b/manual/install.texi index 43dd8d6b81..4bbbfcffa5 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -59,10 +59,21 @@ but the normal setting to install as the standard system library is @samp{--prefix=/usr} for @gnulinuxsystems{} and @samp{--prefix=} (an empty prefix) for @gnuhurdsystems{}. -It may also be useful to set the @var{CC} and @var{CFLAGS} variables in -the environment when running @code{configure}. @var{CC} selects the C -compiler that will be used, and @var{CFLAGS} sets optimization options -for the compiler. +It may also be useful to pass @samp{CC=@var{compiler}} and +@code{CFLAGS=@var{flags}} arguments to @code{configure}. @code{CC} +selects the C compiler that will be used, and @code{CFLAGS} sets +optimization options for the compiler. Any compiler options required +for all compilations, such as options selecting an ABI or a processor +for which to generate code, should be included in @code{CC}. Options +that may be overridden by the @glibcadj{} build system for particular +files, such as for optimization and debugging, should go in +@code{CFLAGS}. The default value of @code{CFLAGS} is @samp{-g -O2}, +and @theglibc{} cannot be compiled without optimization, so if +@code{CFLAGS} is specified it must enable optimization. For example: + +@smallexample +$ ../glibc-@var{version}/configure CC="gcc -m32" CFLAGS="-O3" +@end smallexample The following list describes all of the available options for @code{configure}: @@ -241,7 +252,7 @@ if @code{configure} guesses your machine as @code{i686-pc-linux-gnu} but you want to compile a library for 586es, give @samp{--host=i586-pc-linux-gnu} or just @samp{--host=i586-linux} and add the appropriate compiler flags (@samp{-mcpu=i586} will do the trick) to -@var{CFLAGS}. +@code{CC}. If you specify just @samp{--build}, @code{configure} will get confused. @@ -339,8 +350,8 @@ It is easy to configure @theglibc{} for cross-compilation by setting a few variables in @file{configparms}. Set @code{CC} to the cross-compiler for the target you configured the library for; it is important to use this same @code{CC} value when running -@code{configure}, like this: @samp{CC=@var{target}-gcc configure -@var{target}}. Set @code{BUILD_CC} to the compiler to use for programs +@code{configure}, like this: @samp{configure @var{target} +CC=@var{target}-gcc}. Set @code{BUILD_CC} to the compiler to use for programs run on the build system as part of compiling the library. You may need to set @code{AR} to cross-compiling versions of @code{ar} if the native tools are not configured to work with -- cgit v1.2.3 From e69d994a63afc2d367f286a2a7df28cbf710f0fe Mon Sep 17 00:00:00 2001 From: Zack Weinberg Date: Fri, 29 Jun 2018 16:53:47 +0200 Subject: New configure option --disable-crypt. Some Linux distributions are experimenting with a new, separately maintained and hopefully more agile implementation of the crypt API. To facilitate this, add a configure option which disables glibc's embedded libcrypt. When this option is given, libcrypt.* and crypt.h will not be built nor installed. --- ChangeLog | 23 +++++++++++++++++++++++ INSTALL | 11 +++++++++++ Makeconfig | 9 +++++++-- NEWS | 12 ++++++++++++ config.make.in | 1 + configure | 18 ++++++++++++++++++ configure.ac | 11 +++++++++++ conform/Makefile | 11 +++++++---- crypt/Makefile | 4 ---- elf/Makefile | 27 +++++++++++++++++++-------- elf/tst-linkall-static.c | 4 +++- manual/install.texi | 11 +++++++++++ 12 files changed, 123 insertions(+), 19 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index 9b6edb1f1f..09543433ac 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,26 @@ +2018-06-29 Zack Weinberg + + * configure.ac: New command-line option --disable-crypt. + Force --disable-nss-crypt when --disable-crypt is given, with a + warning if it was explicitly enabled. + * configure: Regenerate. + * config.make.in: New boolean substitution variable $(build-crypt). + * Makeconfig: Only include 'crypt' in all-subdirs and rpath-dirs + when $(build-crypt). + * manual/install.texi: Document --disable-crypt. + * INSTALL: Regenerate. + + * crypt/Makefile: Remove code conditional on $(crypt-in-libc), + which is never set. + * conform/Makefile: Only include libcrypt.a in + linknamespace-libs-xsi and linknamespace-libs-XPG4 + when $(build-crypt). + * elf/Makefile (CFLAGS-tst-linkall-static.c): Only define + USE_CRYPT to 1 when $(build-crypt). + (tst-linkall-static): Only link libcrypt.a when $(build-crypt). + (localplt-built-dso): Only add libcrypt.so when $(build-crypt). + * elf/tst-linkall-static.c: Only include crypt.h when USE_CRYPT. + 2018-06-29 Zack Weinberg * crypt/crypt.h, posix/unistd.h: Update comments and diff --git a/INSTALL b/INSTALL index 052b1b6f89..0a22aa7d01 100644 --- a/INSTALL +++ b/INSTALL @@ -197,6 +197,17 @@ if 'CFLAGS' is specified it must enable optimization. For example: libnss_nisplus are not built at all. Use this option to enable libnsl with all depending NSS modules and header files. +'--disable-crypt' + Do not install the passphrase-hashing library 'libcrypt' or the + header file 'crypt.h'. 'unistd.h' will still declare the function + 'crypt'. Using this option does not change the set of programs + that may need to be linked with '-lcrypt'; it only means that the + GNU C Library will not provide that library. + + This option is for hackers and distributions experimenting with + independently-maintained implementations of libcrypt. It may + become the default in a future release. + '--disable-experimental-malloc' By default, a per-thread cache is enabled in 'malloc'. While this cache can be disabled on a per-application basis using tunables diff --git a/Makeconfig b/Makeconfig index 1afe86475c..608ffe648c 100644 --- a/Makeconfig +++ b/Makeconfig @@ -566,7 +566,7 @@ link-libc-printers-tests = $(link-libc-rpath) \ $(link-libc-tests-after-rpath-link) # This is how to find at build-time things that will be installed there. -rpath-dirs = math elf dlfcn nss nis rt resolv crypt mathvec support +rpath-dirs = math elf dlfcn nss nis rt resolv mathvec support rpath-link = \ $(common-objdir):$(subst $(empty) ,:,$(patsubst ../$(subdir),.,$(rpath-dirs:%=$(common-objpfx)%))) else # build-static @@ -1205,9 +1205,14 @@ all-subdirs = csu assert ctype locale intl catgets math setjmp signal \ stdlib stdio-common libio malloc string wcsmbs time dirent \ grp pwd posix io termios resource misc socket sysvipc gmon \ gnulib iconv iconvdata wctype manual shadow gshadow po argp \ - crypt localedata timezone rt conform debug mathvec support \ + localedata timezone rt conform debug mathvec support \ dlfcn elf +ifeq ($(build-crypt),yes) +all-subdirs += crypt +rpath-dirs += crypt +endif + ifndef avoid-generated # sysd-sorted itself will contain rules making the sysd-sorted target # depend on Depend files. But if you just added a Depend file to an diff --git a/NEWS b/NEWS index ebe63a2f40..7a85a4012c 100644 --- a/NEWS +++ b/NEWS @@ -133,6 +133,18 @@ Deprecated and removed features, and other changes affecting compatibility: binaries. It was just another name for the standard function crypt, and it has not appeared in any header file in many years. +* We have tentative plans to hand off maintenance of the passphrase-hashing + library, libcrypt, to a separate development project that will, we hope, + keep up better with new passphrase-hashing algorithms. We will continue + to declare 'crypt' in , and programs that use 'crypt' or + 'crypt_r' should not need to change at all; however, distributions will + need to install and libcrypt from a separate project. + + In this release, if the configure option --disable-crypt is used, glibc + will not install or libcrypt, making room for the separate + project's versions of these files. The plan is to make this the default + behavior in a future release. + Changes to build and runtime requirements: [Add changes to build and runtime requirements here] diff --git a/config.make.in b/config.make.in index 9e5e24b2c6..d9891b2cd8 100644 --- a/config.make.in +++ b/config.make.in @@ -96,6 +96,7 @@ cross-compiling = @cross_compiling@ force-install = @force_install@ link-obsolete-rpc = @link_obsolete_rpc@ build-obsolete-nsl = @build_obsolete_nsl@ +build-crypt = @build_crypt@ build-nscd = @build_nscd@ use-nscd = @use_nscd@ build-hardcoded-path-in-tests= @hardcoded_path_in_tests@ diff --git a/configure b/configure index 7a8bd3f817..ef18302215 100755 --- a/configure +++ b/configure @@ -676,6 +676,7 @@ build_obsolete_nsl link_obsolete_rpc libc_cv_static_nss_crypt libc_cv_nss_crypt +build_crypt experimental_malloc enable_werror all_warnings @@ -779,6 +780,7 @@ enable_all_warnings enable_werror enable_multi_arch enable_experimental_malloc +enable_crypt enable_nss_crypt enable_obsolete_rpc enable_obsolete_nsl @@ -1448,6 +1450,8 @@ Optional Features: architectures --disable-experimental-malloc disable experimental malloc features + --disable-crypt do not build nor install the passphrase hashing + library, libcrypt --enable-nss-crypt enable libcrypt to use nss --enable-obsolete-rpc build and install the obsolete RPC code for link-time usage @@ -3505,6 +3509,15 @@ fi +# Check whether --enable-crypt was given. +if test "${enable_crypt+set}" = set; then : + enableval=$enable_crypt; build_crypt=$enableval +else + build_crypt=yes +fi + + + # Check whether --enable-nss-crypt was given. if test "${enable_nss_crypt+set}" = set; then : enableval=$enable_nss_crypt; nss_crypt=$enableval @@ -3512,6 +3525,11 @@ else nss_crypt=no fi +if test x$build_libcrypt = xno && test x$nss_crypt = xyes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: --enable-nss-crypt has no effect when libcrypt is disabled" >&5 +$as_echo "$as_me: WARNING: --enable-nss-crypt has no effect when libcrypt is disabled" >&2;} + nss_crypt=no +fi if test x$nss_crypt = xyes; then nss_includes=-I$(nss-config --includedir 2>/dev/null) if test $? -ne 0; then diff --git a/configure.ac b/configure.ac index ca1282a6b3..dc517017f5 100644 --- a/configure.ac +++ b/configure.ac @@ -302,11 +302,22 @@ AC_ARG_ENABLE([experimental-malloc], [experimental_malloc=yes]) AC_SUBST(experimental_malloc) +AC_ARG_ENABLE([crypt], + AC_HELP_STRING([--disable-crypt], + [do not build nor install the passphrase hashing library, libcrypt]), + [build_crypt=$enableval], + [build_crypt=yes]) +AC_SUBST(build_crypt) + AC_ARG_ENABLE([nss-crypt], AC_HELP_STRING([--enable-nss-crypt], [enable libcrypt to use nss]), [nss_crypt=$enableval], [nss_crypt=no]) +if test x$build_libcrypt = xno && test x$nss_crypt = xyes; then + AC_MSG_WARN([--enable-nss-crypt has no effect when libcrypt is disabled]) + nss_crypt=no +fi if test x$nss_crypt = xyes; then nss_includes=-I$(nss-config --includedir 2>/dev/null) if test $? -ne 0; then diff --git a/conform/Makefile b/conform/Makefile index 864fdeca21..74fbda0786 100644 --- a/conform/Makefile +++ b/conform/Makefile @@ -193,13 +193,11 @@ linknamespace-libs-thr = $(linknamespace-libs-isoc) \ $(common-objpfx)rt/librt.a $(static-thread-library) linknamespace-libs-posix = $(linknamespace-libs-thr) \ $(common-objpfx)dlfcn/libdl.a -linknamespace-libs-xsi = $(linknamespace-libs-posix) \ - $(common-objpfx)crypt/libcrypt.a +linknamespace-libs-xsi = $(linknamespace-libs-posix) linknamespace-libs-ISO = $(linknamespace-libs-isoc) linknamespace-libs-ISO99 = $(linknamespace-libs-isoc) linknamespace-libs-ISO11 = $(linknamespace-libs-isoc) -linknamespace-libs-XPG4 = $(linknamespace-libs-isoc) \ - $(common-objpfx)crypt/libcrypt.a +linknamespace-libs-XPG4 = $(linknamespace-libs-isoc) linknamespace-libs-XPG42 = $(linknamespace-libs-XPG4) linknamespace-libs-POSIX = $(linknamespace-libs-thr) linknamespace-libs-UNIX98 = $(linknamespace-libs-xsi) @@ -209,6 +207,11 @@ linknamespace-libs-XOPEN2K8 = $(linknamespace-libs-xsi) linknamespace-libs = $(foreach std,$(conformtest-standards),\ $(linknamespace-libs-$(std))) +ifeq ($(build-crypt),yes) +linknamespace-libs-xsi += $(common-objpfx)crypt/libcrypt.a +linknamespace-libs-XPG4 += $(common-objpfx)crypt/libcrypt.a +endif + $(linknamespace-symlist-stdlibs-tests): $(objpfx)symlist-stdlibs-%: \ $(linknamespace-libs) LC_ALL=C $(READELF) -W -s $(linknamespace-libs-$*) > $@; \ diff --git a/crypt/Makefile b/crypt/Makefile index 303800df73..3811b6e298 100644 --- a/crypt/Makefile +++ b/crypt/Makefile @@ -32,10 +32,6 @@ libcrypt-routines := crypt-entry md5-crypt sha256-crypt sha512-crypt crypt \ tests := cert md5c-test sha256c-test sha512c-test badsalttest -ifeq ($(crypt-in-libc),yes) -routines += $(libcrypt-routines) -endif - ifeq ($(nss-crypt),yes) nss-cpp-flags := -DUSE_NSS \ -I$(shell nss-config --includedir) -I$(shell nspr-config --includedir) diff --git a/elf/Makefile b/elf/Makefile index f221422de3..0eb7c8114e 100644 --- a/elf/Makefile +++ b/elf/Makefile @@ -387,14 +387,21 @@ $(objpfx)tst-_dl_addr_inside_object: $(objpfx)dl-addr-obj.os CFLAGS-tst-_dl_addr_inside_object.c += $(PIE-ccflag) endif -# By default tst-linkall-static should try to use crypt routines to test -# static libcrypt use. +# We can only test static libcrypt use if libcrypt has been built, +# and either NSS crypto is not in use, or static NSS libraries are +# available. +ifeq ($(build-crypt),no) +CFLAGS-tst-linkall-static.c += -DUSE_CRYPT=0 +else +ifeq ($(nss-crypt),no) +CFLAGS-tst-linkall-static.c += -DUSE_CRYPT=1 +else +ifeq ($(static-nss-crypt),no) +CFLAGS-tst-linkall-static.c += -DUSE_CRYPT=0 +else CFLAGS-tst-linkall-static.c += -DUSE_CRYPT=1 -# However, if we are using NSS crypto and we don't have a static -# library, then we exclude the use of crypt functions in the test. -# We similarly exclude libcrypt.a from the static link (see below). -ifeq (yesno,$(nss-crypt)$(static-nss-crypt)) -CFLAGS-tst-linkall-static.c += -UUSE_CRYPT -DUSE_CRYPT=0 +endif +endif endif include ../Rules @@ -1115,7 +1122,6 @@ localplt-built-dso := $(addprefix $(common-objpfx),\ rt/librt.so \ dlfcn/libdl.so \ resolv/libresolv.so \ - crypt/libcrypt.so \ ) ifeq ($(build-mathvec),yes) localplt-built-dso += $(addprefix $(common-objpfx), mathvec/libmvec.so) @@ -1123,6 +1129,9 @@ endif ifeq ($(have-thread-library),yes) localplt-built-dso += $(filter-out %_nonshared.a, $(shared-thread-library)) endif +ifeq ($(build-crypt),yes) +localplt-built-dso += $(addprefix $(common-objpfx), crypt/libcrypt.so) +endif vpath localplt.data $(+sysdep_dirs) @@ -1410,6 +1419,7 @@ $(objpfx)tst-linkall-static: \ $(common-objpfx)resolv/libanl.a \ $(static-thread-library) +ifeq ($(build-crypt),yes) # If we are using NSS crypto and we have the ability to link statically # then we include libcrypt.a, otherwise we leave out libcrypt.a and # link as much as we can into the tst-linkall-static test. This assumes @@ -1425,6 +1435,7 @@ ifeq (no,$(nss-crypt)) $(objpfx)tst-linkall-static: \ $(common-objpfx)crypt/libcrypt.a endif +endif # The application depends on the DSO, and the DSO loads the plugin. # The plugin also depends on the DSO. This creates the circular diff --git a/elf/tst-linkall-static.c b/elf/tst-linkall-static.c index e8df38f74e..d0f2592e67 100644 --- a/elf/tst-linkall-static.c +++ b/elf/tst-linkall-static.c @@ -18,7 +18,9 @@ #include #include -#include +#if USE_CRYPT +# include +#endif #include #include #include diff --git a/manual/install.texi b/manual/install.texi index 4bbbfcffa5..422da1447e 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -230,6 +230,17 @@ libnss_nisplus are not built at all. Use this option to enable libnsl with all depending NSS modules and header files. +@item --disable-crypt +Do not install the passphrase-hashing library @file{libcrypt} or the +header file @file{crypt.h}. @file{unistd.h} will still declare the +function @code{crypt}. Using this option does not change the set of +programs that may need to be linked with @option{-lcrypt}; it only +means that @theglibc{} will not provide that library. + +This option is for hackers and distributions experimenting with +independently-maintained implementations of libcrypt. It may become +the default in a future release. + @item --disable-experimental-malloc By default, a per-thread cache is enabled in @code{malloc}. While this cache can be disabled on a per-application basis using tunables -- cgit v1.2.3 From f2873d2da0ac9802e0b570e8e0b9e7e04a82bf55 Mon Sep 17 00:00:00 2001 From: Florian Weimer Date: Wed, 4 Jul 2018 15:27:24 +0200 Subject: testrun.sh: Implement --tool=strace, --tool=valgrind MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit $(file …) appears to be the only convenient way to create files with newlines and make substitution variables. This needs make 4.0 (released in 2013), so update the requirement to match. Reviewed-by: Carlos O'Donell --- ChangeLog | 11 +++++++++++ INSTALL | 8 +------- Makefile | 55 +++++++++++++++++++++++++++++++++++++++++++++++------ NEWS | 2 +- configure | 2 +- configure.ac | 2 +- manual/install.texi | 8 +------- 7 files changed, 65 insertions(+), 23 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index c17caff776..a8b15be55f 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,14 @@ +2018-07-04 Florian Weimer + + testrun.sh: Implement --tool=strace, --tool=valgrind + * Makefile (testrun-script): Define variable. + (testrun.sh): Use variable. + * manual/install.texi (Tools for Compilation): make 4.0 or later + is required. + * configure.ac: Check for make 4.0 or later. + * INSTALL: Regenerate. + * configure: Likewise. + 2018-07-04 Adhemerval Zanella [BZ #23233] diff --git a/INSTALL b/INSTALL index 0a22aa7d01..3c656fb7a6 100644 --- a/INSTALL +++ b/INSTALL @@ -426,13 +426,7 @@ Recommended Tools for Compilation We recommend installing the following GNU tools before attempting to build the GNU C Library: - * GNU 'make' 3.79 or newer - - You need the latest version of GNU 'make'. Modifying the GNU C - Library to work with other 'make' programs would be so difficult - that we recommend you port GNU 'make' instead. *Really.* We - recommend GNU 'make' version 3.79. All earlier versions have - severe bugs or lack features. + * GNU 'make' 4.0 or newer * GCC 4.9 or newer diff --git a/Makefile b/Makefile index bea4e27f8d..d3f25a525a 100644 --- a/Makefile +++ b/Makefile @@ -128,17 +128,60 @@ ifeq (yes,$(build-shared)) lib: $(common-objpfx)libc.so $(common-objpfx)linkobj/libc.so endif # $(build-shared) +# Used to build testrun.sh. +define testrun-script +#!/bin/bash +builddir=`dirname "$$0"` +GCONV_PATH="$${builddir}/iconvdata" + +usage () { + echo "usage: $$0 [--tool=strace] PROGRAM [ARGUMENTS...]" 2>&1 + echo " $$0 --tool=valgrind PROGRAM [ARGUMENTS...]" 2>&1 +} + +toolname=default +while test $$# -gt 0 ; do + case "$$1" in + --tool=*) + toolname="$${1:7}" + shift + ;; + --*) + usage + ;; + *) + break + ;; + esac +done + +if test $$# -eq 0 ; then + usage +fi + +case "$$toolname" in + default) + exec $(subst $(common-objdir),"$${builddir}", $(test-program-prefix)) \ + $${1+"$$@"} + ;; + strace) + exec strace $(patsubst %, -E%, $(run-program-env)) \ + $(test-via-rtld-prefix) $${1+"$$@"} + ;; + valgrind) + exec env $(run-program-env) valgrind $(test-via-rtld-prefix) $${1+"$$@"} + ;; + *) + usage + ;; +esac +endef # This is a handy script for running any dynamically linked program against # the current libc build for testing. $(common-objpfx)testrun.sh: $(common-objpfx)config.make \ $(..)Makeconfig $(..)Makefile - (echo '#!/bin/sh'; \ - echo 'builddir=`dirname "$$0"`'; \ - echo 'GCONV_PATH="$${builddir}/iconvdata" \'; \ - echo 'exec $(subst $(common-objdir),"$${builddir}",\ - $(test-program-prefix)) $${1+"$$@"}'; \ - ) > $@T + $(file >$@T, $(testrun-script)) chmod a+x $@T mv -f $@T $@ postclean-generated += testrun.sh diff --git a/NEWS b/NEWS index 21b457a050..b1ce067d27 100644 --- a/NEWS +++ b/NEWS @@ -154,7 +154,7 @@ Deprecated and removed features, and other changes affecting compatibility: Changes to build and runtime requirements: - [Add changes to build and runtime requirements here] + GNU make 4.0 or later is now required to build glibc. Security related changes: diff --git a/configure b/configure index ef18302215..eac7f292b4 100755 --- a/configure +++ b/configure @@ -4705,7 +4705,7 @@ $as_echo_n "checking version of $MAKE... " >&6; } ac_prog_version=`$MAKE --version 2>&1 | sed -n 's/^.*GNU Make[^0-9]*\([0-9][0-9.]*\).*$/\1/p'` case $ac_prog_version in '') ac_prog_version="v. ?.??, bad"; ac_verc_fail=yes;; - 3.79* | 3.[89]* | [4-9].* | [1-9][0-9]*) + [4-9].* | [1-9][0-9]*) ac_prog_version="$ac_prog_version, ok"; ac_verc_fail=no;; *) ac_prog_version="$ac_prog_version, bad"; ac_verc_fail=yes;; diff --git a/configure.ac b/configure.ac index dc517017f5..f41ed6decb 100644 --- a/configure.ac +++ b/configure.ac @@ -945,7 +945,7 @@ fi AC_CHECK_TOOL_PREFIX AC_CHECK_PROG_VER(MAKE, gnumake gmake make, --version, [GNU Make[^0-9]*\([0-9][0-9.]*\)], - [3.79* | 3.[89]* | [4-9].* | [1-9][0-9]*], critic_missing="$critic_missing make") + [[4-9].* | [1-9][0-9]*], critic_missing="$critic_missing make") AC_CHECK_PROG_VER(MSGFMT, gnumsgfmt gmsgfmt msgfmt, --version, [GNU gettext.* \([0-9]*\.[0-9.]*\)], diff --git a/manual/install.texi b/manual/install.texi index 422da1447e..42e9954199 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -473,13 +473,7 @@ build @theglibc{}: @itemize @bullet @item -GNU @code{make} 3.79 or newer - -You need the latest version of GNU @code{make}. Modifying @theglibc{} -to work with other @code{make} programs would be so difficult that -we recommend you port GNU @code{make} instead. @strong{Really.} We -recommend GNU @code{make} version 3.79. All earlier versions have severe -bugs or lack features. +GNU @code{make} 4.0 or newer @item GCC 4.9 or newer -- cgit v1.2.3 From e6c695099b7894bce72de04009c889c8f6e674ae Mon Sep 17 00:00:00 2001 From: "H.J. Lu" Date: Wed, 18 Jul 2018 09:52:40 -0700 Subject: Intel CET: Document --enable-cet * NEWS: Mention --enable-cet. * manual/install.texi: Document --enable-cet. * INSTALL: Regenerated. --- ChangeLog | 6 ++++++ INSTALL | 11 +++++++++++ NEWS | 10 ++++++++++ manual/install.texi | 11 +++++++++++ 4 files changed, 38 insertions(+) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index d1c5235849..6d1229ca97 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2018-07-18 H.J. Lu + + * NEWS: Mention --enable-cet. + * manual/install.texi: Document --enable-cet. + * INSTALL: Regenerated. + 2018-07-18 H.J. Lu * sysdeps/x86_64/multiarch/memcmp-sse4.S (BRANCH_TO_JMPTBL_ENTRY): diff --git a/INSTALL b/INSTALL index 3c656fb7a6..844aa0f34c 100644 --- a/INSTALL +++ b/INSTALL @@ -106,6 +106,17 @@ if 'CFLAGS' is specified it must enable optimization. For example: programs and tests are created as dynamic position independent executables (PIE) by default. +'--enable-cet' + Enable Intel Control-flow Enforcement Technology (CET) support. + When the GNU C Library is built with '--enable-cet', the resulting + library is protected with indirect branch tracking (IBT) and shadow + stack (SHSTK). When CET is enabled, the GNU C Library is + compatible with all existing executables and shared libraries. + This feature is currently supported on i386, x86_64 and x32 with + GCC 8 and binutils 2.29 or later. Note that when CET is enabled, + the GNU C Library requires CPUs capable of multi-byte NOPs, like + x86-64 processors as well as Intel Pentium Pro or newer. + '--disable-profile' Don't build libraries with profiling information. You may want to use this option if you don't plan to do profiling. diff --git a/NEWS b/NEWS index c2896a7d93..daef815ae7 100644 --- a/NEWS +++ b/NEWS @@ -9,6 +9,16 @@ Version 2.28 Major new features: +* The GNU C Library can now be compiled with support for Intel CET, AKA + Intel Control-flow Enforcement Technology. When the library is built + with --enable-cet, the resulting glibc is protected with indirect + branch tracking (IBT) and shadow stack (SHSTK). CET-enabled glibc is + compatible with all existing executables and shared libraries. This + feature is currently supported on i386, x86_64 and x32 with GCC 8 and + binutils 2.29 or later. Note that CET-enabled glibc requires CPUs + capable of multi-byte NOPs, like x86-64 processors as well as Intel + Pentium Pro or newer. + * The GNU C Library now has correct support for ABSOLUTE symbols (SHN_ABS-relative symbols). Previously such ABSOLUTE symbols were relocated incorrectly or in some cases discarded. The GNU linker can diff --git a/manual/install.texi b/manual/install.texi index 42e9954199..3a87ac8bb5 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -137,6 +137,17 @@ with no-pie. The resulting glibc can be used with the GCC option, PIE. This option also implies that glibc programs and tests are created as dynamic position independent executables (PIE) by default. +@item --enable-cet +Enable Intel Control-flow Enforcement Technology (CET) support. When +@theglibc{} is built with @option{--enable-cet}, the resulting library +is protected with indirect branch tracking (IBT) and shadow stack +(SHSTK)@. When CET is enabled, @theglibc{} is compatible with all +existing executables and shared libraries. This feature is currently +supported on i386, x86_64 and x32 with GCC 8 and binutils 2.29 or later. +Note that when CET is enabled, @theglibc{} requires CPUs capable of +multi-byte NOPs, like x86-64 processors as well as Intel Pentium Pro or +newer. + @item --disable-profile Don't build libraries with profiling information. You may want to use this option if you don't plan to do profiling. -- cgit v1.2.3 From e84bd8514cd4bf37b37d3f68feafc1e20afa4b56 Mon Sep 17 00:00:00 2001 From: "H.J. Lu" Date: Thu, 19 Jul 2018 12:04:56 -0700 Subject: INSTALL: Add a note for Intel CET status * NEWS: Add a note for Intel CET status. * manual/install.texi: Likewise. * INSTALL: Regenerated. --- ChangeLog | 6 ++++++ INSTALL | 5 +++++ NEWS | 5 ++++- manual/install.texi | 5 +++++ 4 files changed, 20 insertions(+), 1 deletion(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index c40efb6177..32587781b3 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2018-07-19 H.J. Lu + + * NEWS: Add a note for Intel CET status. + * manual/install.texi: Likewise. + * INSTALL: Regenerated. + 2018-07-18 Quentin PAGÈS [BZ #23140] diff --git a/INSTALL b/INSTALL index 844aa0f34c..840b862511 100644 --- a/INSTALL +++ b/INSTALL @@ -117,6 +117,11 @@ if 'CFLAGS' is specified it must enable optimization. For example: the GNU C Library requires CPUs capable of multi-byte NOPs, like x86-64 processors as well as Intel Pentium Pro or newer. + NOTE: '--enable-cet' has been tested for i686, x86_64 and x32 on + non-CET processors. '--enable-cet' has been tested for x86_64 and + x32 on CET SDVs, but Intel CET support hasn't been validated for + i686. + '--disable-profile' Don't build libraries with profiling information. You may want to use this option if you don't plan to do profiling. diff --git a/NEWS b/NEWS index a23d0a7d33..ed61fc5c17 100644 --- a/NEWS +++ b/NEWS @@ -17,7 +17,10 @@ Major new features: feature is currently supported on i386, x86_64 and x32 with GCC 8 and binutils 2.29 or later. Note that CET-enabled glibc requires CPUs capable of multi-byte NOPs, like x86-64 processors as well as Intel - Pentium Pro or newer. + Pentium Pro or newer. NOTE: --enable-cet has been tested for i686, + x86_64 and x32 on non-CET processors. --enable-cet has been tested + for x86_64 and x32 on CET SDVs, but Intel CET support hasn't been + validated for i686. * The GNU C Library now has correct support for ABSOLUTE symbols (SHN_ABS-relative symbols). Previously such ABSOLUTE symbols were diff --git a/manual/install.texi b/manual/install.texi index 3a87ac8bb5..a8577dd610 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -148,6 +148,11 @@ Note that when CET is enabled, @theglibc{} requires CPUs capable of multi-byte NOPs, like x86-64 processors as well as Intel Pentium Pro or newer. +NOTE: @option{--enable-cet} has been tested for i686, x86_64 and x32 +on non-CET processors. @option{--enable-cet} has been tested for +x86_64 and x32 on CET SDVs, but Intel CET support hasn't been validated +for i686. + @item --disable-profile Don't build libraries with profiling information. You may want to use this option if you don't plan to do profiling. -- cgit v1.2.3 From f650932b34f23b94a49b245405db65c3b00bd0ed Mon Sep 17 00:00:00 2001 From: Carlos O'Donell Date: Tue, 31 Jul 2018 16:37:07 -0400 Subject: Update tooling versions verified to work with glibc. --- ChangeLog | 3 +++ INSTALL | 33 +++++++++++++++++++++++---------- manual/install.texi | 31 +++++++++++++++++++++---------- 3 files changed, 47 insertions(+), 20 deletions(-) (limited to 'INSTALL') diff --git a/ChangeLog b/ChangeLog index dc409ae17d..5e78ad8ce1 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,8 @@ 2018-07-31 Carlos O'Donell + * manual/install.texi: Update versions. + * INSTALL: Regenerate. + * manual/contrib.texi (Contributors): Update contributions. 2018-07-31 Carlos O'Donell diff --git a/INSTALL b/INSTALL index 840b862511..781cb8415b 100644 --- a/INSTALL +++ b/INSTALL @@ -301,23 +301,28 @@ for testing, which run together with the rest of the testsuite through 'make check'. These scripts require the following tools to run successfully: - * Python 2.7.6/3.4.3 or later + * Python 2.7/3.4 or later - Python is required for running the printers' test scripts. + Python is required for running the printers' test scripts. As of + release time, Python 3.6 is the newest verified to work to test the + pretty printers. * PExpect 4.0 The printer tests drive GDB through test programs and compare its output to the printers'. PExpect is used to capture the output of GDB, and should be compatible with the Python version in your - system. + system. As of release time PExpect 4.3 is the newest verified to + work to test the pretty printers. - * GDB 7.8 or later with support for Python 2.7.6/3.4.3 or later + * GDB 7.8 or later with support for Python 2.7/3.4 or later GDB itself needs to be configured with Python support in order to use the pretty printers. Notice that your system having Python available doesn't imply that GDB supports it, nor that your - system's Python and GDB's have the same version. + system's Python and GDB's have the same version. As of release + time GNU 'debugger' 8.0.1 is the newest verified to work to test + the pretty printers. If these tools are absent, the printer tests will report themselves as 'UNSUPPORTED'. Notice that some of the printer tests require the GNU C @@ -444,12 +449,15 @@ build the GNU C Library: * GNU 'make' 4.0 or newer + As of relase time, GNU 'make' 4.2.1 is the newest verified to work + to build the GNU C Library. + * GCC 4.9 or newer GCC 4.9 or higher is required. In general it is recommended to use the newest version of the compiler that is known to work for building the GNU C Library, as newer compilers usually produce - better code. As of release time, GCC 7.3 is the newest compiler + better code. As of release time, GCC 8.1.1 is the newest compiler verified to work to build the GNU C Library. For PowerPC 64-bits little-endian (powerpc64le), GCC 6.2 or higher @@ -476,7 +484,7 @@ build the GNU C Library: You must use GNU 'binutils' (as and ld) to build the GNU C Library. No other assembler or linker has the necessary functionality at the - moment. As of release time, GNU 'binutils' 2.29.1 is the newest + moment. As of release time, GNU 'binutils' 2.31.1 is the newest verified to work to build the GNU C Library. * GNU 'texinfo' 4.7 or later @@ -493,7 +501,7 @@ build the GNU C Library: 'awk' is used in several places to generate files. Some 'gawk' extensions are used, including the 'asorti' function, which was introduced in version 3.1.2 of 'gawk'. As of release time, 'gawk' - version 4.2.0 is the newest verified to work to build the GNU C + version 4.2.1 is the newest verified to work to build the GNU C Library. * GNU 'bison' 2.7 or later @@ -505,13 +513,15 @@ build the GNU C Library: * Perl 5 Perl is not required, but it is used if present to test the - installation. We may decide to use it elsewhere in the future. + installation. We may decide to use it elsewhere in the future. As + of release time 'perl' version 5.28.0 is the newest verified to + work to build the GNU C Library. * GNU 'sed' 3.02 or newer 'Sed' is used in several places to generate files. Most scripts work with any version of 'sed'. As of release time, 'sed' version - 4.4 is the newest verified to work to build the GNU C Library. + 4.5 is the newest verified to work to build the GNU C Library. If you change any of the 'configure.ac' files you will also need @@ -521,6 +531,9 @@ and if you change any of the message translation files you will need * GNU 'gettext' 0.10.36 or later + As of release time, GNU 'gettext' version 0.19.8.1 is the newest + version verified to work to build the GNU C Library. + You may also need these packages if you upgrade your source tree using patches, although we try to avoid this. diff --git a/manual/install.texi b/manual/install.texi index a8577dd610..c39e63bf3b 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -336,23 +336,26 @@ These scripts require the following tools to run successfully: @itemize @bullet @item -Python 2.7.6/3.4.3 or later +Python 2.7/3.4 or later -Python is required for running the printers' test scripts. +Python is required for running the printers' test scripts. As of release time, +Python 3.6 is the newest verified to work to test the pretty printers. @item PExpect 4.0 The printer tests drive GDB through test programs and compare its output to the printers'. PExpect is used to capture the output of GDB, and should be -compatible with the Python version in your system. +compatible with the Python version in your system. As of release time +PExpect 4.3 is the newest verified to work to test the pretty printers. @item -GDB 7.8 or later with support for Python 2.7.6/3.4.3 or later +GDB 7.8 or later with support for Python 2.7/3.4 or later GDB itself needs to be configured with Python support in order to use the pretty printers. Notice that your system having Python available doesn't imply that GDB supports it, nor that your system's Python and GDB's have the same -version. +version. As of release time GNU @code{debugger} 8.0.1 is the newest verified +to work to test the pretty printers. @end itemize @noindent @@ -491,13 +494,16 @@ build @theglibc{}: @item GNU @code{make} 4.0 or newer +As of relase time, GNU @code{make} 4.2.1 is the newest verified to work +to build @theglibc{}. + @item GCC 4.9 or newer GCC 4.9 or higher is required. In general it is recommended to use the newest version of the compiler that is known to work for building @theglibc{}, as newer compilers usually produce better code. As of -release time, GCC 7.3 is the newest compiler verified to work to build +release time, GCC 8.1.1 is the newest compiler verified to work to build @theglibc{}. For PowerPC 64-bits little-endian (powerpc64le), GCC 6.2 or higher is @@ -522,7 +528,7 @@ GNU @code{binutils} 2.25 or later You must use GNU @code{binutils} (as and ld) to build @theglibc{}. No other assembler or linker has the necessary functionality at the -moment. As of release time, GNU @code{binutils} 2.29.1 is the newest +moment. As of release time, GNU @code{binutils} 2.31.1 is the newest verified to work to build @theglibc{}. @item @@ -541,7 +547,7 @@ GNU @code{awk} 3.1.2, or higher @code{awk} is used in several places to generate files. Some @code{gawk} extensions are used, including the @code{asorti} function, which was introduced in version 3.1.2 of @code{gawk}. -As of release time, @code{gawk} version 4.2.0 is the newest verified +As of release time, @code{gawk} version 4.2.1 is the newest verified to work to build @theglibc{}. @item @@ -555,14 +561,16 @@ verified to work to build @theglibc{}. Perl 5 Perl is not required, but it is used if present to test the -installation. We may decide to use it elsewhere in the future. +installation. We may decide to use it elsewhere in the future. As of release +time @code{perl} version 5.28.0 is the newest verified to work to build +@theglibc{}. @item GNU @code{sed} 3.02 or newer @code{Sed} is used in several places to generate files. Most scripts work with any version of @code{sed}. As of release time, @code{sed} version -4.4 is the newest verified to work to build @theglibc{}. +4.5 is the newest verified to work to build @theglibc{}. @end itemize @@ -580,6 +588,9 @@ and if you change any of the message translation files you will need @itemize @bullet @item GNU @code{gettext} 0.10.36 or later + +As of release time, GNU @code{gettext} version 0.19.8.1 is the newest +version verified to work to build @theglibc{}. @end itemize -- cgit v1.2.3